Visual-Meta

Visual-Meta is a method of including meta-information about the document and its contents visibly in the document as a human and machine readable appendix. It contains citation, addressing and formatting information.

Immediate User Benefit

Visual-Meta enables Augmented Copying (Copying As Citation, or  Scholarly Copy,) by providing a transparently easy way to add full metadata to documents (initially PDF). Proof-of-concept implementations are Author and Reader as shown here in a brief demo: https://youtu.be/rjeEPnPzD6c

Enables Rich Views & Interactions

Adds rich metadata of the formatting of included elements including headings for instant folding of the document and executing searches with heading elements included in the results, and description of how to parse tables, images and special interactions such as graphs, for dynamic re-creation by reader software.

Not Yet-Another-Standard-Proposal [It’s BibTeX & JSON]

This is not a new format, this is a novel use and extension of the academic-standard BibTeX format with JSON additions.

Furthermore, the approach is not tied to existing means of dealing with information, it is simply an approach and should always lean away from optimising what is presented and towards making it easily human and machine understandable, including the addressing mechanisms used, which should not simply be server-centric, but more robust and include redundancy (Voß, 2019). This will also help us deal with link-rot and other needs for maintain with server based models in the future (Anderson, Carr, Millard, 2017). Further developments can support different types of metadata, including semantic metadata (Al-Khalifa & Davis, 2006), yet it does not impose a new standard, such as the semantic web does (Marshall & Shipman, 2004), while it at the same time opens new opportunities for Visual-Meta aware systems, providing immediate benefits, as outlined above.

Unleashes Hypertextuality

As it is today, academic documents have a few special fields for metadata (Abstract and Keywords) but they are not included in Reference Section of the documents which cite them, hence they are one step removed for analysis and not available to the reader. Visual-Meta can easily accommodate such extra metadata without interfering with the fashionable cosmetic layout preferences of the academic field, institution or journal. This can allow documents can take on hypertextual node characteristics. The benefits can be profound.

A description of this is available at wordpress.liquid.info/07/unleashing-hypertextuality-in-documents/frode/

Wider, Deeper Benefits

Visual-Meta provides robust support for advanced interactions by storing meta at content level. Visual-Meta stores dynamic interactions in a non-interactive medium. Visual-Meta can also provide servers with information about what is in the document in a semantically meaningful way for better extraction of textual and multimedia components.

A Visual-Meta PDF document will be able to survive in a hybrid digital-analog environment (Laouris, 2015) and through changes in technological infrastructures for as long as documents can be printed and the PDF document model will be understood. A Visual-Meta document can be printed, then scanned again and with OCR all the benefits of Visual-Meta will be available again, reducing the need for elaborate link re-creating interventions (Morishima, Nakamizo, Iida, Sugimoto, Kitagawa, 2009) (Kolak & Schilit, 2008). Because all the interactable variables, can potentially be recorded in the Visual-Meta, this is a path to full, not partial (Marshall & Golovchinsky, 2004), archivability of interactive text with explicit knowledge presented and included (Carr, Miles-Board, Woukeu, Wills, Hall, 2005). It could also become a powerful tool in analysis and operations of multiple documents, where links could be based on inferable relationships between attributes of a document (Carr, 2007), truly releasing the potential power of digital metadata (Tarrant, Carr, Payne, 2008) and the utility of digital ‘eprint’ repositories (Hitchcock, Carr, et al., 2004).

ACM Hypertext 2019 Visual-Meta Presentation

Key Benefits : Why Visual Vs Embedded

  • Advanced Meta embedded in the document header or package is not directly accessible by end user
  • Easy to Add & Extract. A common complaint about embedded meta is that there is no standard beyond the basics (which are not often employed) and is therefore near-impossible to use at scale. Being based on BibTeX means that a simple copy and paste will add significant added, useful meta
  • Self-explaining standard which requires no technical expertise to add
  • End-User immediate benefit for adding Visual-Meta. End-users who add Visual-Meta to their own or legacy PDFs have the immediate benefit of Scholarly Copy and not being locked into a Reference Manager, making Visual-Meta more adoptable than trying to establish a new header-meta standard.
  • Robust:
    • Can survive document format change
    • Can survive printing out and scanning and OCR and nothing is lost
    • All supported meta can survive document format and operating system updates without becoming unreadable
  • Trivially easy for a human reader to verify
  • Trivially easy to append to legacy documents and to strip if not desired anymore
  • Can handle large amounts of formatting information for reader software to use to reformat and re-present the document as well as provide rich interactions

Exceptionally Easy To Add To Legacy Documents

Legacy documents can easily have Visual-Meta appended upon being opened in a Visual-Meta aware PDF reader for use immediately or in the future. 49 Second demonstration of how to apply Visual-Meta to any document which has a DOI to allow Copy As Citation: youtu.be/ymtoOnPH0A4

Legacy Academic Document Support

Legacy documents can easily have Visual-Meta appended upon being opened in a Visual-Meta aware PDF reader for use immediately or in the future.

49 Second demonstration of how to apply Visual-Meta to any document which has a DOI to allow Copy As Citation: (youtu.be/ymtoOnPH0A4)

Example

This is what the Visual-Meta for the Visual-Meta ACM article looks like. Please note, font and size does not matter. The formatting has been co-designed with Jakob Voß at github.com/nichtich/visual-meta

author = {Hegland, Frode},
title = {Visual-Meta: An Approach to Surfacing Metadata},
booktitle = {Proceedings of the 2Nd International Workshop on Human Factors in Hypertext}, series = {HUMAN '19},
year = {2019},
isbn = {978-1-4503-6899-5},
location = {Hof, Germany},
pages = {31--33},
numpages = {3},
url = {http://doi.acm.org/10.1145/3345509.3349281},
doi = {10.1145/3345509.3349281},
acmid = {3349281},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {archiving, bibtex citing, citations, engelbart, future, glossary, hypertext, meta,
metadata, ohs, pdf, rfc, text}, }
@{visual-meta-end}
  • Addressing Information

Addressing information for citing the document is the usual citation information (author, title, etc.) and will have scope to be augmented with high resolution (Wilde & Baschnagel, 2005) linking to web pages, blogs in particular and in-PDF sections and robust multi-addressing. This is ongoing work which can strengthen the peer-to-peer connectivity document (rather than server or location) addressability can offer (Wiil, Bouvin, Larsen, De Roure, Thompson, 2004).

  • Formatting Information

The formatting specification is implemented as custom fields, which can include anything the authoring software can describe, for extraction into interactive systems. Please also look at the JSON Extension below.

General Formatting: formatting = { heading level 1 = {Helvetica, 22pt, bold}, heading level 2 = {Helvetica, 18, bold}, body = {Times, 12pt}, image captions = {‘Times, l4, italic, align centre} },

Citation Formatting, to allow reader application to display citations in any style:citations = { inline = {superscript number}, section name = {References}, section format = {author last name, author first name, title, date, place, publisher} },

  • Glossary, to allow reader application to see any use glossary terms:

glossary = { term = {Name of glossary term}, definition = {freeform definition text}, relates to = {relationship – “other term”},  term = {Name of glossary term number two}, definition = {freeform definition text}, relates to = {relationship – “other term”}, },

  • Special

Special, to allow the authoring application to add anything, which a human programmer or advanced ML can read and optionally use:

special = { name = {DynamicView}, node= {nodcname, location, connections} }

  • Provenance

The ‘version’ field is the version of Visible-Meta, the ‘generator’ is what added the Visual-Meta to the document and the ‘source’ is where the data comes from, particularly to be used if appended to a legacy document:

visible-meta = { version = {1.1}, generator = {Liquid | Author 4.6}, source = {Scholarcy, 2019,08,01} }

  • @End Tag

Please note, the ‘@{visual-meta-end}’ is crucial to have as the last element in the document since it is recommended to parse the document in reverse and have the software look for this element to confirm that visual-meta is present.

Computational Text Extensions

Extensions can include time information, location information and Vint Cerf introduced the term ‘Computational Text’ summer of 2020 to cover text in a document which can be integrated with by the author’s choice, rather than by brute force in reader software. This is the type of work Bret Victor (Victor, 2010) is known for and Bruce Horn has developed decades ago. It takes on new relevance with Visual-Meta and Reader since the aspect of Visual-Meta being ‘reverse CSS’, in the words of Jakob Voß. Text or images in the body of the document can be referred to in the Visual-Meta and instructions for interactions presented. This is an early overview of computational text types, which will be elaborated on by the Future Text Initiative advisors over time at computationaltext.info

• Relative time dimensions such as ‘yesterday’, ‘tomorrow’, ‘Friday’ and so on could be stored with reference to the date and time it was typed, if the author chooses, for future reading to either automatically update the text or to allow a reader to manually or by preference specify that any dates should be from the reader’s point in time.
• Any mathematical equations, although tools external to the text can make this work.
• Names of people, places etc. can be encoded with multiple spellings or writing styles (with or without middle initial fx) and external identity servers, such as ORCID.
• Alternative versions for authoring, so that an author can toggle which paragraph or word should be used (like Final Cut Auditions).
• Images to replace text in specific views, such as company logos in a graph view or in a scrolling/search view.
• Glossary definitions which can be dynamically expanded and contracted in the text when reading, at the reader’s presence.
• Transclusions and live links to external text.
• Links which contain the entirety of what they link to, for robustness, in addition to the link.
• Geographical information behind text such as ‘here’.
• Actual computer code, if computer language is somehow and somewhere specified.

The first part shows what text in the document is referred to, followed by the date and finished with a description of what type of data it is:

“In text Reference”  “Data”  “Type of Data/location of description”

<json>
[ {“name”:”8:23am, Tuesday, 13th of May 2020″, “2,208,988,800”:”typeNTP”},
{“name”:”14th & Madison, NY”, “Latitude: 40 degrees, 42 minutes, 51 seconds N”:”latlong”},
{“name”:”David Millard”, “0000-0002-7512-2710, “person”:”https://orcid.org”},
] </json>

Please keep in mind that the goal is to be able to copy and paste data across systems while specifying how it is defined and formatted, as shown in brackets above. This is about self-declaring data (to a human or translation code) visible in plain sight, it is about allowing users to copy and paste self-defining JSON data.

These can be data pods, they do not necessarily need to be part of a document. As for the specifics, that is purely a matter of implementation.

Picture the scenarios: You read an ordinary PDF and you come across the time an event happened and you can click on that time and a menu of options are presented (depending on what reading software you are using), including showing exactly how long ago it was in the past (or how near in the future) and lets you copy the time and use it when you come across another time event where you can now automatically see how far apart they are in time.

Picture the same with geographical information; you can copy and paste locations and use them semantically with other locations.

Imagine coming across the names of people and having a solid link to their online presence and not having to guess who is really who.

And much, much more–this addressability creates the opportunity for rich, useful interactions.

Rights Extension

There is no reason why the Visual-Meta cannot encode the rights the author confers onto the reader, for use, re-use, transclusion and caching.

JSON Extension for Headings

JSON can be used to augment the way headings are recorded for a more robust result, as used in The Future of Text book:

[ {“name”:”Acknowledgements”, “level”:”level1″},

{“name”:”Contents”, “level”:”level1″},

{“name”:”Dear Reader of The Distant Future”, “level”:”level2″},

{“name”:”Dear Reader of Today”, “level”:”level2″},

{“name”:”the future of text : Articles”, “level”:”level1″},

{“name”:”Adam Cheyer”, “level”:”level2″},

{“name”:”Adam Kampff”, “level”:”level2″}, }]

JSON can also potentially be used to encode the entire document to enable advanced functions like complete reformatting of the document to suit the reader. Since the visual-meta can be very, very small, this does not have to impact the document page number significantly.

‘Stamp’ Of Provenance

When a citation is done through Visual-Meta, the references can Visual-Meta represents a break in how citations are handled, by mandating that they be included in the document at the same level as the content of the document itself, leading to a solid chain from document to citation. Because this is an entirely new level of security and robustness when citing, I suggest that citations made in this way be somehow tagged in the Reference list to show its Visual-Meta origins. Maybe something like this:

[24] Kitromili, S. & Jordan, J. & Millard, D., in Proceedings of the 30th ACM Conference on Hypertext and Social Media. 2019.
What is Hypertext Authoring? New York, NY, USA.
DOI: 10.1145/3342220.3343653. {Visual-Meta}

Future Text Initiative

The Visual-Meta approach is part of the Future Text Initiative which also includes the book The Future of Text and the Author, Reader and Liquid software projects.

Further Information

Further description is on the blog: wordpress.liquid.info/visual-meta and further information at: Visible-Meta Example & Structure. Full source code for parsing visual-meta will be made available here. Addressing is discussed at wordpress.liquid.info/10/scholarly-copy-addressing-clipboard/frode/

The visual-meta approach is very much inspired by Doug Engelbart’s notion of an xFile and his insistence that high-resolution addressability should be human readable. Here is an brief interview with him from the early 2010s, with more available on thefutureoftext.org/engelbart.html#ohsxfiles

‘This is a very important concept’
Vint Cerf