Re-Appraising the Appraisal Tab

On Sunday, October 10, 2021, I gave a talk for Archivematica Con. This is an adaptation of the talk. You can see my slides, if you prefer.

---

As the archives of the University of Michigan, the Bentley Historical Library acknowledges that the historical origins and present locations of the University were made possible by cession of lands by Anishinaabe and Wyandot peoples under coercive treaties common in the colonization and expansion of the United States. We note in particular the grant of land made by the Anishinaabeg under the Treaty of the Foot of the Rapids, for the “college at Detroit,” so that their children could be educated. These lands continue to be the homeland of indigenous people, and we seek to reaffirm and respect their contemporary and ancestral ties to the land and to recognize their contributions to the University.

Many of you will be aware that between 2014 and 2016 (i.e., five–five!–years ago), the Bentley Historical Library’s Mellon Foundation-funded ArchivesSpace-Archivematica-DSpace Workflow Integration project integrated these three Open Source Software platforms to expedite the ingest, description, and overall curation of digital archives. It did this by facilitating the creation and reuse of descriptive and administrative metadata among what, at the time, were “emerging” platforms, and streamlining the deposit of fully processed content into a digital preservation repository.

ArchivesSpace-Archivematica-DSpace Workflow Integration

This is something we’ve talked about a lot in the past, so I won’t go into too much detail here. But, for some important context for the remainder of this presentation, the Bentley contracted with Artefactual Systems, the developers of Archivematica, for development work in the following three areas:

1. The creation of a new Appraisal tab in Archivematica that permitted users to characterize, review, arrange, and describe digital archives with such features as identifying file format distributions (in both tables and pie charts, as shown here), and many more.

1. The “tightly-coupled,” as I would now call it, and, despite our best efforts, very prescriptive, integration of Archivematica and ArchivesSpace, so that users could:

• Review, create, and edit archival description from ArchivesSpace directly within Archivematica (with information being written back to ArchivesSpace via its API) without having to switch between applications/browser windows.
• Drag-and-drop content from the Backlog pane onto archival description in the ArchivesSpace pane, thereby associating data with metadata (and also establishing a SIP ready to undergo Archivematica’s Ingest procedures).

1. The, again, relatively tightly-coupled integration of Archivematica and DSpace so that the Archivematica Storage Service splits the AIP into two archive files (one for the digital content, and the other for administrative metadata and log files, which would be restricted from public access by default) and automatically deposits them as two bitstreams on a new item to the selected DSpace collection.

We’ve been using Archivematica in production for ~5 years, and in that time we’ve trained ~20 archivists, project archivists, and students on this workflow integration and have used Archivematica to process ~10 TB of born-digital “mixed materials.” I’ve included two charts one of my colleagues recently put together because I think it’s interesting to see trends over time, and to see how things have gone during the pandemic when it comes to digital and physical processing.

The Broader Effects of a Major Grant Project

That’s all well and good, but coming onto our “6th+” year of using Archivematica in production I wanted to talk a bit about the broader effects of a major grant project, at least at the Bentley, the state of Archivematica (and the Appraisal tab and this workflow integration) today, some technical lessons learned, and, because “technical challenges in archives are now better understood also as cultural and social challenges” (thank you, Mark Matienzo), an attempt at some cultural and social lessons learned, as well.

So, first, the broader effects of a major grant project, starting with some of the effects we started seeing even in the middle of doing the grant project itself. Or, as my colleague Dallas (who was originally a co-presenter on this presentation!) and I called it at the time on this very flashy title slide…

…the “brighter the lightning, the louder the thunder.”

“Working Smarter” (in Gigantic Scare Quotes)

First, it taught us the value of “working smarter” (in gigantic scare quotes because I don’t always think that “working smarter” in the colloquial sense is always, actually, working smarter) by starting to learn some basic technical skills, at least as individuals. This was in part because of needing to learn and implement Archivematica, which isn’t necessarily for the faint of heart, and in part because of things like a major metadata clean up project that we had to undertake as a prerequisite for making use of the workflow integration we developed. When we learned that ⅓ (~1,000) of our EADs wouldn’t import out of the box into ArchivesSpace for any number of reasons, we quickly realized that we needed to find ways to approach remedying this problem at scale.

So, we began to develop basic technical skills in-house, again, mostly as individuals. Early on, we focused a lot on “scripting it” with Python, but I’d throw into the mix getting comfortable on the command line, working with databases, parsing XML, using version control, and interacting with APIs as important complementary technical skills we developed at the time. Needless to say, this ability to work smarter has had a very positive impact on many other areas of our work.

Making Tech Skills a Strategic Priority

Working smarter as individuals led us to thinking more about making technical skills a strategic priority. For institutions that lack internal IT support or want to avoid a situation where one individual is responsible for all technical issues, a key question is how to foster the development these skills as a group, especially when the vast majority of the professional literature on the topic of acquiring and/or expanding technical skills assumes that the onus of technical upskilling lies with the individual archivist or librarian. Which, it doen’t have to!

So, at the Bentley, we have put focused effort into group learning around technical upskilling, developing the technical skills of staff and at the same time expanding the Bentley’s capacity to address core duties and responsibilities. These have ranged from informal workshops, to an intensification of this effort with more emphasis on an inclusive approach to peer-to-peer learning experiences within the Curation team (and a mandate to assess outcomes and identify opportunities for future projects), to generating deliverables around topics that directly relate to our work, e.g., the development of a Python-based command-line tool to automate QC for digitized audio.

This process has also made us more collectively aware of the limitations of our own silos of expertise and the ways that the digital initiatives we undertake impact the overall curation of archival materials.

The “Agile” Archivist (Scare Quotes, Again)

During the grant project, we also learned about agile software development, as it is (or at least was) the software development methodology used by Artefactual. I’m sure you’re aware that agile values individuals and interactions with them over processes and tools, working (i.e., good enough) software over comprehensive documentation (although documentation is also important), customer collaboration over contract negotiation, and responding to change over following a plan.

In our work with Artefactual, I feel like we “gleaned” a lot from them when it came to agile software development. At the time, I think I overly focused on the “agile” part; today I can safely say that this experience set us on a trajectory that ultimately led us to a better understanding of project management more generally and its importance to leading initiatives and getting things done in an organization. And, just as with the development of technical skills, the development of project management skills has also had a positive impact in many other areas of our work.

From Project Management to Strategic Thinking About Projects

Over the years, we have continued to cultivate and harvest this kind of “agile” seed that Artefactual helped plant in our minds. As I said, our initial fixation with “agile” gradually grew into an exploration of project management concepts more generally. That exploration of project management gradually led to thinking more strategically about what projects we want to undertake in the first place (and how to prioritize them and schedule them), including by using frameworks like Divergent-Emergent-Convergent Thinking for brainstorming exercises and Strategy Knotworking for strategic thinking. And that thinking more strategically has led to looking for ways to make that process more inclusive and participatory, including with techniques like Liberating Structures that are designed to turn things like traditional organizational hierarchies and “expert at the front of the room” modes of thinking upside down. I’m very proud of this and this has been another important effect of the grant project.

…

In years 2-5, some important organizational changes have also occurred, but before we get into that I want to get back into the state of Archivematica, especially the Appraisal tab and workflow integration, at the Bentley, today.

Archivematica, the Appraisal tab, and the Workflow Integration Today

So… to make a long story short.

As it stands now, the Appraisal tab is no longer functional, the DSpace integration is no longer functional, and the UI crashes regularly. We have not been successful upgrading to a more current and officially supported release. After talking with our colleagues in the LIT department of U-M Library (from whom we are administratively separate, but who had been providing technical support for Archivematica for many years out of what I think might best be described as “goodwill” or “being a good neighbor”) and with our own administration (who control the budget), we’re currently in talks with Artefactaul about an annual maintenance agreement.

Re-Appraising the Appraisal Tab (and the Workflow Integration)

The slightly longer version…

Re-Appraising the Workflow Integration

After the completion of the ArchivesSpace-Archivematica-DSpace Workflow Integration project, we used the Appraisal tab and the workflow integration extensively. Over time, however, we found ourselves moving away from this “tightly-coupled” integration. (Actually, that’s what this presentation was initially about… but a lot has changed in the two years since we submitted the proposal!) It turns out it works really well for relatively small, heterogeneous content, but in our experience does not work as well for relatively large or homogenous content (or content destined for access systems that aren’t DSpace, like Kaltura which we use for streaming A/V). For better or worse, this relative “largeness,” homogeneity, or even non-mixed material”ness” has come to characterize more and more of our accessions or at least the way we approach processing them.

Instead, we developed our own Python scripts (remember those technical skills?) to replicate a lot of what Archivematica was doing, but outside of Archivematica, which we think provides more flexibility. For example, we have created a Python ArchivesSpace wrapper (BHL ASpace Client), and Python DSpace wrapper (DAPPr), and even “AIP Repackaging” scripts (BHL AIP Repackager that make use of those wrappers to repackage Archivematica AIPs that we create via the Automation Tools, deposit them to DSpace, and update ArchivesSpace with the handles, all in batch and what in feels like an incredible economy of scale. And now we do this for pretty much everything, even the small, heterogeneous content. So we still have an integration, it’s just more what I would call “loosely-coupled.” (I promise, I didn’t know what Sarah was going to be talking about in terms of tightly-coupled and loosely-coupled organizations.) That is, the systems aren’t talking directly to each other, and the workflow isn’t so prescriptive, but we’ve still found a way to make the “hand-off” of data and metadata between them as flexible and automated as possible.

Re-Appraising the Appraisal Tab

And, yes, this has also meant that we have moved away from doing appraisal inside Archivematica to doing appraisal outside of Archivematica. Most recently we have used Archivematica solely for creating AIPs (I take it that was always what it was supposed to do anyway) with the Automation Tools and some CSVs pointing to associated records in other systems instead of using it for managing a backlog or for appraisal or for managing Archival Storage, and that seems to have been a good development, at least for us.

Archivematica Technical Support

Meanwhile, we have had some staff turnover at both the Bentley and the LIT department of our University Libraries. The latter, especially, has resulted in a lack of internal knowledge of Archivematica and its specialized integrations with other applications in our digital preservation environment. This has had a significant impact on our ability to get technical support for the application.

In May, we started to run into some Archivematica issues that weren’t “relatively straightforward to resolve.” These were the words of the head of U-M Library LIT… with whom we have a very loose agreement–i.e., not formalized or written down… you know, the kind that always works out in the end–to provide Archivematica technical support for issues up to and including “relatively straightforward to resolve,” but nothing more.

We are currently running Archivematica 1.9.2 (so we are a bit behind the curve and definitely not on a currently supported version) with Automation Tools. There are about five users at the Bentley, but not all of us interact with the dashboard (and, in fact, since those issues arose, none of us can anyway), choosing instead to kick off transfers using Automation Tools.

(This, by the way, I feel like is worth saying, has worked just fine, even though this required some specialized technical knowledge on our part, which, because of the grant, we had. While all this bad stuff was happening, Archivematica happily chugged away at TBs worth of digital material kicked off via the Automation Tools over the last year and a half or so.)

So… maintenance is hard, and as I said, after waffling a bit about some one-time training, we’re currently in talks with Artefactaul about an annual maintenance agreement (like MOMA, I guess).

…

To be clear, even once we get these Archivematica technical support issues resolved, I don’t know if we’ll go back to using the Appraisal tab or the workflow integration, at least as they were conceived of five years ago and are currently implemented, for the reasons I listed above. While they did provide an easy on ramp for digital processing for those archivists who aren’t familiar with digital preservation or don’t know how to use a CLI, it’s also true that the Appraisal tab and the workflow integration just don’t “fit” our current circumstances as well as the Automation Tools and a “loosely-coupled” integration do. (15:20)

Technical Lessons Learned

So, what have we learned? I want to start with some technical lessons we learned. The first one, and I’d be interested to hear what someone from Artefactual or others have to say about this, especially with the “product” or “value” way of working in the Archivematica Product Support Program is about scope creep, systems integration, and the Appraisal tab.

Scope Creep, Systems Integration, and the Appraisal tab

So, I’ll just come out with it: I have often questioned our decision to add appraisal functionality to Archivematica, and this is even though we spent the majority of our time during the grant project, and money, on developing the Appraisal Tab, even to the detriment of other development tasks like DSpace integration. I worry that we were trying to make a system accomplish a goal widely outside the scope of its original design. I don’t think Archivematica was really designed to do appraisal; it was designed to create AIPs. So, there are ways in which the Appraisal tab feels like a kind of “add-on.”

To make matters, worse, in practice, it was never really that useful for processing archivists anyway, and the kind of technical or feasibility appraisal it enables (based on the more technical outputs of file format identification and characterization tools) turned out to be only moderately useful for archivists focused on digital preservation and less so for processing archivists needing to do intellectual appraisal of content. In hindsight, it’s amazing how useful a directory listing is, even a top-level one, especially for large accessions. There really aren’t that many “digital junk drawers” out there, at least in my experience. Needless to say, there are much easier ways than the Appraisal Tab to get such a directory listing.

So, to quote “No Silver Bullet,” I worry that this is an example of “arbitrary complexity, forced without rhyme or reason by the many human institutions [like U-M] and systems to which [the software developers of Archivematica] interfaces must conform,” one of the “essential difficulties” of software engineering. In fact, I’ve sense learned that’s the whole point of systems integration–the approach we took with the rest of the ArchivesSpace-Archivematica-DSpace workflow integration project–to let each platform in a technical ecosystem specialize in one function or task but work together with other systems that specialize in other functions as a coordinated whole. That might have been a better approach to appraisal in or with Archivematica.

Maslow’s “Hierarchy of Needs” as Applied to Systems Integration

As I have thought about systems integration more generally, I’ve begun to think about a kind of Maslow’s “Hierarchy of Needs” as applied to building and maintaining an integrated technical ecosystem. I’m sure you’re all familiar with the “Hierarchy of Needs.”

First, we have the basic needs for bodily functioning (or wifi); then there is the desire to be safe, and secure in the knowledge that those basic needs will be fulfilled in the future too. The next stage is all about social recognition, status and respect. As each level of needs is satisfied, the desire to fulfil the next set kicks in. If things like basic needs are not met, then you can’t really focus on anything else.

I take it that there are serious reservations with this model from fields of both psychology and management. But I couldn’t help but try to map this in terms of technical lessons learned about systems integration. So, here we go.

• At the bottom, you have resources, human and financial, although I’d actually say that people are most important here and at every step along the way. I’ll get into this more in a second, but ideally these are or become “regular” archivists (i.e., not term-limited) and recognizing that grant or otherwise “soft” money only goes so far and should at least eventually become hard money for core operations.
• After that, it’s important to have tools or systems in place like Archivematica (and, in our case, ArchivesSpace, DSpace, and Kaltura). Ideally these are designed to “play nicely” with other systems.
• Then, it’s technical support for those systems. A lesson we learned recently and is probably a residual of the “project” beginnings of our adoption of Archivematica.
• Then, I think you get to this kind of sweet spot of basic technical (and project management) skills–the kind I described above–and “loosely-coupled” systems integrations that aren’t overly prescriptive or force you into a particular workflow. You can do a lot here in this zone, and for us hanging out here has been truly transformational, even when tightly-coupled integrations fail.
• And, finally you have tightly-coupled systems integration. Like self-actualization in Maslow’s actual “Hierarchy of Needs,” it’s nice, but not necessary, and definitely fragile.

[An Attempt at] Social and Cultural Lessons Learned

So, very quickly, because I’m new to thinking about this kind of thing, here we go:

Resources

• It seems important that “project archivists” that were involved with this digital preservation project eventually became “regular” (i.e., permanent, tenure track archivists) dealing with digital preservation on a more regular, permanent basis. This is like the “project” to “program” point Sarah was making yesterday. It makes me happy that administrators were able to make the connection between “digital” and “core operation” and “not going away” at the time (in fact, I benefited from this), and these days I, as a kind of middle manager, am trying to help them also recognize the connection between “analog and physical” and “core operation” and “not going away” and, by the way, “also impacts digital.”
• This reasoning also applies to the way that we moved from soft money (i.e., the grant project, relatively unstable money, which paid for even my own salary during my first year) to General Fund money, which is taken from relatively stable sources of income like tuition and fees (which pays for my salary now).

Organization

• When I think about the issues we encountered with technical support, I am actually sympathetic toward our U-M Library LIT colleagues. They don’t have any incentive to support Archivematica! There are so few people on campus, all of whom happen to currently work at the Bentley, who actually use this system. Compare that with something like that catalog or with other enterprise-wide systems with central ITS more generally, which many, many more people use on a daily basis. Even though Archivematica is very vital for us, overall I can understand the incentive problem this might create for them.
• “Good will” in organizations only goes so far. You know, we have not had any formal agreement with U-M Library to support Archivematica (or other “archives” systems like Aeon and ArchivesSpace that they have, at various points in time, also supported). We’ve had some individual people that are interested in things like digital preservation, and, God help them, METS, but when they leave, so does Archivematica technical support. So, we’ve slowly but surely been exploring how feasible it might be to support these systems on our own or at least without them, in part because U-M Library has slowly but surely been honest with us in telling us what they can and can’t actually do. As I mentioned, for Archivematica, we’re exploring technical support with Artefactual. But we also now completely host Aeon on our own, using ITS servers and some support from Atlas, and over the summer we started hosting a development environment for ArchivesSpace as a kind of experiment into what we might actually be able to realistically support internally.
• When we can’t do something “on our own,” we’re also working with, I think, important University stakeholders like the General Counsel’s office and the Provost to develop MOUs that clearly define our relationships with campus partners beyond the kind of goodwill that individuals often extend to one another, but which doesn’t last.

Pandemic and the “Great Resignation”

• And, yes, people do leave, and, yes, a pandemic makes people rethink their careers, work conditions, and long-term goals (and leave). There have been pros of being “good to go” with digital processing when it came to the pandemic. Because of Archivematica, we’ve been able to support remote work with digital processing (you saw the spike in 2020-2021), and that was even without a full-time digital archivist or even an Archivematica GUI that worked and, um, a pandemic. But, the reason we didn’t have a full-time digital archivist was because ours left, first for another institution for personal reasons during the pandemic, and then because he left the profession, in part because of “the great resignation” and remote work.
• And, so, while we also happen to have the counterexample working at the Bentley (an IT person who, during midlife, decided to become an archivist), we’re not immune to the trends of the “Great resignation,” or even of people in archives learning technical skills and then leaving the profession for what they perceive are greener, more technical pastures. For example, we definitely don’t pay as much as private enterprise… recruiting and employment for people with these kinds of technical skills in libraries and archives is challenging.

Ok! That was too much. Please let me know if you have any questions. Thank you.

Categories: talks