LOST IN THE DATA PORTAL.

Digitization of cultural heritage is still a recent development. Metadata digitization, however, has its roots as early as the 1960s when MARC (Machine Readable Cataloguing) revolutionised cataloguing practices in the library field. Computers have made standardization and sharing of metadata within the library, archive and museum fields a lot easier but also a lot more complex. The library field was already familiar with a long tradition of sharing records for many other institutions the necessity of sharing their records across the field has not been a common practice (Skinner 53-4). Since the last century many efforts have gone into the standardization of metadata across the field resulting in various different schemes and models, depending not only on the type of collection or object different descriptions would be given but also on the purpose such as retrieval, records and descriptive. But not only the different efforts of standardization but also the introduction of automated/digital management systems that require yet another format have let many institutions to have not just collections of heritage object but collections of metadata each in a different schema. Users have always relied on the metadata records for their research as it helped them identify whether or not the object or items holds the information they seek. Although searching the records can sometimes feel like a daunting task as users have to come into the institution for their quest they can always rely on the help and guidance of experts archivist, librarians or curators to find that right collection to move them forward.

When institutions started to use the internet to display their collections, portals and platforms provided the opportunity for users to engage with cultural heritage from the comfort of their home. For the first time collaboration across the cultural heritage field on a large scale was made possible by integrating all these online collections on one website. All these different metadata schemas and the diversity in formats had to come together and the messiness of the data was made painfully clear. As users are coming less to the institutions and rely more and more on the online availability of records user studies have been crucial to evaluate the quality of the online collections in portals and platforms and to discover what is expected and how people are engaging with cultural heritage online. Each portal and each project attempting to unify the chaos promises to be the ultimate solution yet a formal evaluation methods lack and when funding runs out many projects are abandoned leaving even more different formats. User studies have always been very good at pointing out that metadata should be improved yet they do not offer institutions any guidance on how to identify which collections are most at risk of being unusable due to bad data or recommendations on how to improve data in a way that both suits institutions and users.

This paper seeks out to describe how users are using online collections on platforms and portals, what they are expecting to better understand what user’s need are. Followed an institution’s perspective on what is being done to improve data quality and a look at crowdsourcing as a possible cost-effective solution for data creation and improvement. Finally, this paper proposes a framework for to evaluate the data quality of a collection while comparing the situation with other institutions for possible data sharing opportunities.

As researchers are coming less into heritage institutions to search their collections the face to face help that the institutions offer dwindles to the background. Instead, ‘[p]eople expect to find archives and special collections on the open Web using the same techniques they use to find other things, and they expect comprehensive results. The invisibility of archives, manuscripts and special collections may well have more to do with the metadata we create than with the interfaces we build.’ (Schaffner 86). Metadata, the data about the data, has been crucial for management of information whether they are stored on punch cards, written on paper, organized in excel sheets or contained in a management system they have been the go-to source in order to discover where or what items are held in a collection. Yet their creation is often driven by different philosophies and approaches which per field and per institutions even per person can differ widely. Digitization of metadata records has exposed the discrepancies in the records as well as fired up the debates about what the focus should be when creating metadata. Back in 1988 Dowler already advocated the need to adjust practices to suit not the theories but the users of the archive (74-80). He describes in his paper the different approaches between librarians and archivist when they view themselves as gatekeepers of their collections. Although both set out to make their collections more accessible their ideas about how this should be done are quite different: librarians seek to make the users self-sufficient in their search whereas archivist like to guide ‘a grateful scholar by the hand through the uncharted forest of the record to precisely the right material’ (Dowler 82). That Dowler advocates for more user-focused studies at the end of the eighties seem logical for us now as users reach institutions through different channels which are not always easy to identify what their purpose is.

Having still all users coming physically through the door observing their searching behaviour and aiding them in their quests is relatively straightforward. Real-life presence of someone knowledgeable from the institutions self is approachable for all sorts of questions from researchers. They also have plenty of opportunities to talk to the researchers, ask them more about the research and their needs as well providing more guidance towards collections and items that the researchers self wouldn't initially have thought of. This counts for librarians as well even though they initially take a more hands-off approach they are still physically there for any inquiries. Both archivist and librarians work as gatekeepers of information, their in-depth knowledge of the collections and their decisions on how collections are formed and described influences researchers hugely yet at the same time if anything is unclear those from the institutions aiding researchers are the first to know.

Dowler's plea, unbeknown to himself, comes from the introduction of the electronic recorders systems that made its way into the reading rooms and started to replace the more traditional card catalogues and microfiches. Even though he states that archivist has little knowledge on who their users are even before any introduction of electronic records management systems the ease of finding out who the users are and how they are searching the archives is relatively easy as described above.

Our approach to searching for information has drastically changed since the introduction of computers and the internet which confirmed Dowler's view that more attention should be given to the actual users. The field has listened, the go-to tool to evaluate both systems and collections is through user studies and over the years they have provided valuable insights on who the users are, how they search, what their expectations are and what needs to improve. Unfortunately, they do often lack the how. Published research on how to improve is often linked to funded projects that either develop tools, host platforms or portals and commercial companies who offer solutions for a price, they all come with a mild or non-existing critical evaluation. Those working with collections in the heritage field are often confronted with limited funds, high workloads little time and an enormous backlog. Improving records and creating them often comes down to routine procedures that have been in place long before the current employees started working for the institution that often serves one single goal: reducing the amount of backlog as fast as possible (O'Hara Conway and Proffitt 19-20).

The pull for digitization of cultural heritage comes both from the users as from the institutions itself even governments are a pushing factor behind the digitization. Records, catalogues and metadata have been one of the first items to be digitized due to the fact that it is flat text and the usefulness for internal institutional use. The introduction of computers may have made user studies more complex but it also can increase the access to the heritage collections and open up a wider audience. Digital records also enabled institutions to share their data with each other and the world which in essence would make it easier to supplement each other and share best practices but it has also shown the huge amount of discrepancies, messiness in the data. Issues concerning records and especially metadata of individual items are taking almost half of the time allocated to digitization projects making it an important and costly investment (Primary Research Group 28).

Digitization of the records and metadata have made the users obscure, hidden behind screens and IP-addresses and more difficult to research. To better allocate resources for digitations and the evaluation of past projects user studies have made a very important contribution to the understanding of how digital collections are being found and used. “The more users do not need to consult archivists and librarians for searching, the more successful initiatives to improve description and discovery have been” (Schaffner 87). Interestingly, although most user studies are great at pointing out where digital collections are failing to meet user’s expectations they often do not provide the strategical or methodological how to improve the digital collections. This, however, does not take away the immense valuable insights that we have learned about user’s online activity over the past twenty to thirty year, especially for institutions that are just embarking on the digitisations of their collections. Metadata forms a critical part in the discovery and understanding of heritage items and collections, badly formatted or lack of metadata will result in online hidden collections as they can hardly be found. Knowing what is expected can help with the initial designs and setting up digitization projects. In an extensive literature review of user studies on special collections in both libraries and archives, Schaffner identifies several important characteristics of researchers and their interactions with the digital collections:

“Data is not necessarily meant to be understood in a context other than the knowledge domain it was created in and the purpose for which it has been published. Lots of information is therefore implicit for the professionals who created the data, but becomes less obvious once published within end-users’ applications or interfaces” (‘We Want Better Data Quality’).

Over the years many collaborations of institutions have led to a multitude of different formats that were to supposed to solve system integration problems as well as provide constituent formatted metadata for researchers across the institutions. However, the large variety of metadata schemata with mindboggling acronyms have contributed more to the problem of inconsistent data rather provide a solution (Park and Tosaka 109). Although calls for a more common metadata format remain due to the enormous backlog and the push to have collections out on the web a radical change in the creation of metadata seems to be necessary. “Cultural institutions tend to think about data quality in isolation, however, the users of the data may be in the best position to comment on the data and help improving it” (‘We Want Better Data Quality’). Crowdsourcing may be the way forward for creating and improving metadata. Although their might be a fear among cataloguers and records creators that crowdsourcing will leave them without a job and the content that will be created through the crowd will be of such poor quality that the effort of collecting them cost time and money then it saves. Crowdsourcing is however sometimes seen as an evil no use waste of institutions monies but largely these negativities are based on myths that through thorough research and experiments have been largely debunked. Crowdsourcing has been in place for quite some time and its quality closely monitored through experiments and evaluations (Aroyo and Welty 16).

Metadata enrichment and corrections are not a new thing, OCR corrections were one of the first and easiest crowdsource projects as it simply involves reading written or old printed text and compare the output and correct it where necessary. One of the biggest yet low covert projects for OCR corrections is Google’s reCAPTCHA that used sections from Google Books that either had trouble or were not able to be transcribed automatically for the no robot validations on websites. Unknowingly millions of people contributed better descriptions for Google Books while trying to sign into websites, order items, sign up for newsletters. As more people fill in the same word for a reCAPTCHA that word is adopted as the correct transcriptions, although there are some that had even human eyes didn’t understand “reCaptcha nevertheless achieves an accuracy rate above 99 percent, which compares favourably with professional human transcribers” (Gugliotta).

It must be noted however, that reCAPTCHA received many critiques on the fact that people were not aware that they were working on transcriptions while not receiving any compensation. This argument let in 2015 to a lawsuit against Google for stealing people’s time as the transcription was used to for books that Google sold, however, in the following year the judges dismissed the case as the transcription was also used for other free services of Google as well the triviality about the amount of time consumed (Harris, Kravets).

Users studies have provided insights into how users are engaging with online cultural heritage collections, they no longer access institutions websites, portals and platforms directly, but rather use the open web with keyword and subject search to find what they need and believe to a certain extent that online collection is equal to those that are offline. Researchers want better data quality and more collections online even if it is just records, while institutions would like to meet all these expectations they are under serious budget strains and are facing enormous backlogs. Crowdsourcing can offer a solution for institutions to tackle their own problems and meet researcher’s needs while at the same time engaging with the wider public and increasing the institution’s exposure.

Once collections are out on the web they are knowing and unknowingly being picked up and reused by portals, platforms, APIs, social media and blog, it is no surprise that heritage institutions are not fully aware of what data of theirs is out there and how the quality is. User’s studies only point out that the data needs to be improved but In order to identify which collections are most at risk of not being found and used due to poor data quality they do not provide any guidance or method how to tackle this problem. This method will expose discrepancies in the data by tracing a ‘single’ object across different institutions and platforms. Although institutions in the cultural heritage field like to boast about the unique objects they have in their collections they also have a lot of objects of which (semi-)duplicates or similar versions can be found in other collections of other institutions as well, sometimes even within the same collection. These objects can be key in measuring the level synchronicity of the metadata online with those available in the institution self and have the ability to trace it back from the platform to the institution. In an ideal situation we would know exactly which objects are present in which institutions and collections and be able to only digitize one and have all the others refer to the same digitized object and supplement the records, it would be immensely cost saving but it also requires organizational collaboration effort on a global scale which at the moment is still very far away. With the urge to digitize material now and it being more cost-effective to do entire collections rather than small selections the likelihood of the same/similar object being digitized by different institutions is great. (Primary Research Network 33) Although one ‘digital copy’1 representing all different versions out there might not be the ultimate solution if every single object should have its own digital counterpart the data should be roughly the same. Right now we only discover these multiple copies when the different collections come together online, especially within portals as they often contain collections from archives, libraries, museums and sometimes even galleries.

1 Personally I do not consider the digitized version of a real life object to be a copy or even a 2D/digital representation. The identity of the digitized object is far more problematic as it is stuck between two dimensions: it is both a shadow of its real life counterpart as it is an entirely new object in itself. As this requires more extensive explanation for which this paper is not well suited nor does it help to clarify the method, therefore, I will be using more generally accepted terms such as digital ‘copy’ or ‘version’.