Calculating electronic records storage costs

Last week, I walked through the various considerations and costs of storing paper records.  This post will do the same for electronic records and follows the same formula of not taking into account personnel or overhead costs or depreciation of equipment.  If you prefer a truncated version of this information, I’ve created a 1-page brochure of questions to consider about electronic records storage costs.

The first steps are the same, whether you store your electronic records on premise or in the cloud:

  1. Calculate the amount of storage necessary for your electronic records:
    1. image of folder propertiesFor born-digital records, this can easily be determined from the properties of existing file folders.  If your organization tends to retain records for the long-term or is not in the habit of purging routine, obsolete, and trivial data, this number will tend to grow exponentially.
    2. If you plan to scan paper records, you’ll need to determine the file format and resolution you intend to employ before you can estimate the necessary storage.  (The Library of Congress maintains information about recommended file formats.)
  2. Multiply the answer found in #1 by 2 so you can have at least one backup of all your files.

On-Premise Storage

Calculating the costs for storing your electronic records on premise will largely depend on the size of your organization.  Larger organizations likely have IT departments that set fee scales for each gigabyte (GB) of storage used.  Be aware:

  • the cost may differ depending on whether you wish to store sensitive or non-sensitive data
  • there may be minimum blocks of storage available
  • the cost is likely calculated based upon the storage available to you, not the amount of storage you actually use
    sample costs for server storage

In smaller organizations, you can calculate the cost of each GB of storage by dividing the capacity into the cost.  But don’t be surprised if the advertised capacity of the drive is different than what the computer registers — most humans think in base 10 numbers, while the computer calculates in base 2.summary of drive capacityFrom least to most expensive, storage options include magnetic data tape, hard drive disks, and solid state drives (although the initial set-up costs for tape storage will probably exceed the other options).  If you want to be able to find your records, you should also invest in a document management system or some other means of indexing your electronic records.

Cloud Storage

Cloud storage means that your data is stored on remote servers rather than on premise, making it accessible through the Internet rather than a direct connection.

  1. Cloud storage generally breaks down into four components (and many providers have pricing calculators):
    1. Storage, calculated on the amount of data stored and usually charged at monthly rates
    2. Requests (e.g., put, copy, list, select) and data retrieval
    3. Data transfers (e.g., your website calls an object that’s in cloud storage)
    4. Data management (e.g., inventory, analytics, object tagging, replication)
  2. Determine whether the cost of backing up your files is included with the above fees or requires additional payment.
  3. Confirm the vendor’s process for purging data that has met its required retention (including any replications) as well as what sort of destruction certification they provide.  For anything other than standard storage, (e.g., nearline, coldline, archival), you will probably be charged for a minimum storage period, even if you choose to delete the file sooner.
  4. Determine whether there is proprietary software necessary to access your stored records and whether it is included in the storage fees or requires additional seat licenses.

Although these won’t directly impact your costs, here are some additional considerations that should be addressed in your contract with a cloud storage vendor:

  • Is there a system provided for indexing the records?
  • Is there a mechanism for avoiding spoliation of evidence (i.e., a method for establishing a litigation hold)?
  • What are the performance/availability guarantees (e.g., planned and unplanned downtime)?
  • Who owns the data that is stored and does the vendor have the right to access your data and profit from it in any way?
  • What is the procedure (and cost) for exporting records (including images as well as metadata) at the end of the contract period and/or when vendor ceases operation?
  • If your organization is subject to compliance requirements, you probably want to determine specifically where your vendor will be physically housing your electronic records (e.g., are they stored in Europe and, therefore, now subject to GDPR requirements?).
  • If your organization works in a highly regulated field like law enforcement or healthcare, you also will want to investigate the enforcement of security provisions by the vendor.
  • If possible, you may even want to determine what power source your vendor uses to cool the server farm, so you can take into consideration the environmental impacts of your electronic storage.

Hybrid solution

In reality, you may wind up with a combination of on-premise and cloud storage for your electronic records (e.g., you may store your active records on premise and backup your records to the cloud).  If your employees do not all have Virtual Private Network (VPN) access to locally stored files, this may be a factor that encourages more usage of cloud storage (this is definitely the teleworking influence talking!).

The complicated part of calculating the costs of electronic records storage is not in the formulas but is instead in the various factors, like:

  • Do you need the capability of sharing files with people outside your organization?
  • Will you digitize records into TIFF images or PDF files?
  • Which files can be relegated to coldline storage because they are only rarely accessed?
  • Do you have the staff who can adequately monitor and maintain on-premise storage systems?

But I have included a few calculations in the spreadsheet that might prove useful.  (As with the paper records calculators, the yellow cells are places where you should enter data; the remaining cells have formulas.)

I’ll conclude with one sound piece of records management advice — just because storage is relatively cheap, don’t let your organization plan to retain all electronic records permanently.  There are risk management and administrative reasons for defensible destruction that should outweigh the financial considerations.

Advertisement

Calculating paper records storage costs

Periodically on the Records Management Section listserv, a question arises about how to calculate the costs of records storage.  Sometimes the person is doing budget planning or looking to contract services; others may be formulating an argument about the value of good records management.  So I decided to make this my project for RIM month and have developed some simplistic models for calculating the costs of storing records, both paper and electronic.  (Please note: these models do not take into account personnel or overhead costs or depreciation of equipment.)

If you prefer a truncated version of this information, I’ve created a 1-page brochure of questions to consider about paper records storage costs.  What I’ll do here is walk you through the models and provide examples for calculating the costs of storing paper records.  Information about storing electronic records will follow in a separate post.

In-House File Cabinets

If you’re setting up an office or reconfiguring your space, it might be useful to calculate how much it costs to have file cabinets occupying some of that space.

  1. Identify the number of cabinets used/needed for storage.  This is a simple count of existing cabinets.  To calculate how many cabinets you may need, a standard 4-drawer vertical letter-size file cabinet holds about 6 cubic feet of records.
  2. Calculate the number of file folders needed to house the records (at least 1 folder for every inch of paper).  A standard drawer has about 26 inches of usable space, so for ease of calculation, you could estimate at least 30 folders per drawer.
  3. Calculate the cost of each file cabinet.
    1. One-time cost to purchase the cabinet
    2. For leased space, ongoing cost to use floor space for cabinets — measure footprint of cabinet + space that must be left free in order to open drawers and access records stored
    3. For owned space, there’s also a less monetarily-quantified opportunity cost if you’re using floor space for cabinets rather than other purposes (e.g., could a file room be turned into a break room or a conference room?)

In order to compare effectively with other options, I’ll calculate the monthly cost of each cubic foot of records stored in a file cabinet in your office (with the yellow cells being places you fill in the information you gathered and the others having formulas you can see here):calculator for file cabinet costs

In-House Storage Center

You may own a warehouse or similar sort of facility that you can use as a storage center.  For the purpose of this exercise, I’ll assume the records have already been foldered (but if not, see above for calculating expense).

  1. Calculate in cubic feet the volume of records to be stored.  The University of Delaware Archives and Records Management provides a useful conversion chart.  Or another way to look at it is that a standard cubic foot box has dimensions of 15”x12”x10” and about 3,000 pieces of letter-sized paper can fit inside (although this number will be dramatically lowered depending on the use of fasteners and folders).
  2. Calculate the number of boxes needed to house the records.  (HINT: Should be the same answer as #1!)  Determine the cost of each box.
  3. Calculate the number of shelving units needed for storage.  The specifications for the shelving unit should indicate the dimensions, and if you’re looking at shelving actually intended for shelving boxes of records, they’ll probably even list how many boxes can be stored on each unit.
  4. Calculate the amount of floor space necessary for the shelving units and to access the boxes stored on the units.  Estimate at least 3 times the footprint of the shelving unit.  As with the above example, for leased space, you’ll have the ongoing cost to use this floor space for shelving.calculator for storage center costs

This model assumes the real estate used for this storage center is much less valuable than your office space.  If you want to be thorough in your budgeting calculations, you should also identify the service equipment that will be necessary to store records (e.g., ladders, carts, pallet jack).  And if you want to be able to find boxes that you store, you should also buy/develop an inventory system.

Both of these in-house models calculate the up-front costs (e.g., cabinets, boxes) into the total monthly costs.  If you want to see a longer term method for calculating costs once these are paid, go to the spreadsheet.

Off-Site Vendor

This model begins much the same as storing records in your own storage center, but there are significant differences that you should make sure are explicit in your contract, such as security and privacy guarantees.  Also make sure you understand all potential fees (e.g., administrative fees, delivery fees, etc.).

  1. Calculate in cubic feet the volume of records to be stored.
  2. Calculate the number of boxes needed to house the records.
  3. Identify the cost to deliver records to off-site storage.
  4. Identify the fee to store records (probably monthly).
  5. Estimate the cost to retrieve and replace records (probably set fee per box):
    1. Consult internal usage stats and/or consider the likelihood of audits/litigation/records requests for stored documents.
    2. May also include transportation costs to and from storage facility.
    3. Be aware there’s also a fee to refile the pulled records.
  6. Identify the cost to destroy the records.calculator for vendor costs

Whether you need to make plans to store paper records or you need to encourage defensible destruction as a cost saving measure, I hope these models will be useful to you.  And if you’re interested in seeing the considerations for storing electronic records, come back next week!

RIM month during a state of emergency

Records and Information Management month was founded by ARMA International in 1995, and it has been a focal point of activities for this Records Management Section of SAA for a long time.  But this is not a typical April.  The latest numbers I saw this morning are that at least 87% of people in the U.S. are under stay-at-home orders.  Most archives and businesses cannot carry out business as usual because of the unprecedented health and economic impacts of COVID-19, so we certainly don’t want to give the impression that we are calloused to these hardships.

But for a number of reasons, I don’t want to forgo this opportunity to highlight the importance of RIM work:

  • I can scarcely think of a situation where information has been more vital to our well-being.  I heard Jon Meacham’s evaluation this morning that “to a large extent . . . the Enlightenment’s on trial here.  Facts and data that shape human decisions because they are objectively true, that is something that is now very much under assault.”  So what can we as archivists and records managers be doing to ensure true facts and data are being collected and disseminated?
  • People who have never before talked about data are now touting the importance of tracking who has the coronavirus and who has developed antibodies, all of which can shape medical and policy decisions moving forward.  Are there RIM voices involved in these conversations to ensure the appropriate methods for collecting and maintaining this data as well as the necessary privacy and confidentiality protections?
  • Many people who are not accustomed to it are being forced to telework — all the while juggling numerous other family and financial concerns — so good records management may not be at the top of their list of priorities.  What can the RIM community do to ease the burden while also maintaining good (if not best) practice?
  • On a related point, many of us don’t have in close proximity the human sources to whom we usually turn for guidance, so we’re using technology to fill in those gaps — and creating lots of records where there ordinarily would be none.  Do all our constituents know how to handle these MS Teams chats and videoconferencing recordings etc.?
  • Even in the midst of this crisis, there is essential work like processing payroll and setting up utilities that must be accomplished.  Have our Continuity of Operations Plans been thorough enough to be responsive in the current situation?
  • At some point, we’ll emerge on the other side of this crisis and will need the ability to learn from what was handled well and what was disastrous.  Are we capturing the appropriate documentation of policy and personal impacts of this crisis?
  • Outside the pandemic, there are two other pivotal data collection efforts ongoing — the 2020 Census and primary elections.
    • There were already digital options in place for collecting census data, but traditionally there have also been census workers who go door-to-door to capture information from people not willing or able to submit their data electronically.  The impacts on funding and representation will be dramatic if this data is incomplete.
    • Some primaries have already been postponed, and some locales are looking into the possibility of voting options that are not in-person.  Are there RIM people involved in setting up procedures to guarantee the authenticity of these tallies?

So both on this blog and on the SAA RMS listserv, you will continue seeing RIM month communications.  Some, like the conversation starter that was posted on the listserv yesterday, will be crisis-specific in hopes of providing people with opportunities to reflect and share how their lives and work are being impacted.  Some may be more lighthearted, knowing we could probably all use an opportunity to forget our troubles for a few minutes.  And some will be very practical, like the information on calculating records storage costs that I’ve been teasing you with for months.  My hope is that these efforts will provide you with some useful information, some relevant suggestions, and a mechanism to stay engaged with your community despite our social distancing.  I challenge you to weigh in either here or on the listserv with your thoughts.  And as always, if you have direct requests for the Records Management Section steering committee, you can reach us at saarecordsmanagement@gmail.com.  Stay healthy!

Courtney Bailey (Chair, SAA Records Management Section)

RIM implications of teleworking

Over the past weeks, more and more businesses, government agencies, educational facilities, and cultural heritage organizations have shuttered their physical locations — either in a proactive attempt to prevent community spread of the coronavirus or in response to local shelter in place/stay at home orders — and have stood up teleworking and online options.  As people involved with records and information management, we realize that the location at which folks are doing their work has no bearing on the record status of the files created.  But in recognition that good RIM practices may not be on the forefront of many people’s minds during this crisis, allow me to call attention to some things to consider (many of which were contributed through the SAA Records Management Section listserv).

  • Rather than trying to maintain paper records outside the office in non-secure locations, try to conduct as much business electronically and take as few paper files home as possible.  Also try to print as little as possible.  Obviously, any paper files that are needed during this time should be maintained appropriately and returned to the office as soon as practicable, either to be filed or shredded as appropriate.
  • For the purposes of records management, public records requests, audits, etc., to the greatest extent possible, employees should maintain files within agency-managed environments (e.g., SharePoint, OneDrive); if these resources are not available, files that are maintained on personal devices must be transferred to agency resources as soon as practicable.  In the meantime, auditors, general counsel, and public information officers should provide guidance about how records requests will be handled. 
  • Publicly available Wi-Fi systems are not secure, so data security protocols demand that any confidential agency documents should not be accessed while using public Wi-Fi.
  • If documents with personal identifying information or other confidential matters are handled in a home office, make sure they are not available to other parties.  If they need to be destroyed before you can return to your business office, they should be shredded rather than placed in the trash.

There are already some good resources available — although they predate this situation, they certainly address many of the issues of teleworking:

There are also questions that remain about the impacts of this situation:

  • Good Continuity Of Operations Plans should include information about essential records.  As early as 2015, the Alabama Department of Archives and History listed pandemic influenza training and exercises in its COOP template — I wonder if anyone had undertaken exercises before this current crisis that prepared your institution and employees?  If so, it’d be interesting to find out how effectively your training translated to actual deployment and what, if anything, you wish you’d done differently.
  • What happens when people need access to paper files that are in remote storage?  I haven’t seen these businesses specifically identified as essential in the various government orders, but can a legal argument be made that they are?
  • Does anyone already have procedures in place about how records requests will be handled when employees are working remotely and may or may not be capturing all records in agency platforms in real time?

For the longer term, it will also be interesting to see how this crisis affects people’s definitions of essential records and organizations’ procedures for guaranteeing access to essential records.  In a wider scope, is there a way for workers to identify the electronic version of a go-kit (i.e., the electronic files that need to be accessible remotely if teleworking is required, even if they are not technically essential)?

If anyone has any stories to share, please comment here or post to the listserv through SAA Connect.  And if you prefer not to share your comments publicly, you can always reach out to the Records Management Section directly, and we can anonymize/aggregate this information for the benefit of the whole community.

Archivists and Records Managers, part 11

At the 1965 annual meeting of the Society of American Archivists, SAA Fellow and National Archives director Frank Evans delivered a paper about the relationship of archivists and records managers.  He recapped the literature of the previous two decades (much of which has already been cited in this series) and provided this conclusion about the intertwining roles of archival and records management work and the necessity of collaborating with allied professions:

“The interest of the archivist in records management is therefore not only legitimate—it is essential.  Conversely, it is the recognition and full acceptance of his responsibilities in these matters that distinguish the professional records manager.  Like the archivist he too is ultimately responsible to society at large and thus to posterity.

     “Regardless of the particular routes we may travel in our need for professional betterment, we share the common problems of the need for education, training, and closer relations with all of our colleagues in the fields of information and documentation.”

Archivists and Records Managers, part 10

Charles M. Dollar worked as the electronic records program manager for NARA from 1974-1994, and during this time, he delivered a number of lectures and papers that coalesced into an article published in the Fall 1993 issue of Archivaria entitled “Archivists and Records Managers in the Information Age.”  He put forward the simple argument that handling electronic records should force archivists and records managers to remember their common concerns — specifically, “records integrity, records disposition, and records accessibility” — and work together to create answers to the twists and turns electronic records created in their fields.  He identified the necessity of baking records appraisal and disposition into IT applications so that obsolete and irrelevant records are not migrated to new systems.  He concluded with a challenge for archivists and records managers not only to work together but also to expand their reach:

“It is not enough that archivists and records managers agree upon a joint agenda and talk about it.  There must also be aggressive activities that carry archivists and records managers into the main stream of the information management community.”

Archivists and Records Managers, part 9

Jay Atherton was a long-time archivist at the Public Archives of Canada.  In the Winter 1985-86 issue of Archivaria, he published “From Life Cycle to Continuum: Some Thoughts on the Records Management – Archives Relationship.”  Atherton emphasized the necessity of cooperative work between records managers and archivists to accomplish certain goals:

“- ensure the creation of the right records, containing the right information, in the right format;
– organize the records and analyze their content and significance to facilitate their availability;
– make them available promptly to those (administrators and researchers
alike) who have a right and a requirement to see them;
– systematically dispose of records that are no longer required; and
– protect and preserve the information for as long as it may be needed (if
necessary, forever).”

He concluded, “A symbiotic relationship between an archivist and a records manager should facilitate the achievement of these ends.  The intellectual training and historical perspective of the archivist will enrich the practical, immediate concerns of the records manager.  And the records manager’s knowledge of his institution, as well as his concern for efficiency, practicality, and immediate service, will help the archivist to perform his responsibilities.  Working as a team within the records management-archives continuum, they will ensure that their ultimate goals – administrative and cultural – are achieved.”

Archives*RM Testimonial #6

This testimonial about the intersections of archives and records management comes from Lori Eaton, Archivist at Found Archives, LLC.

At a meeting of foundation archivists in June 2019, Darren Walker, president of the Ford Foundation, spoke about the value he found in the margin notes past presidents had made on letters, reports, board materials and other documents preserved in the foundation’s archives. His comment has nagged me ever since.

I work as a consulting archivist helping foundation staff who are charged with managing how their organization is administered. Though they may have different job titles, these are typically the folks who create and update records management policies and retention schedules (or bring me in to help them with these tasks). In contemporary foundations, this means working with born-digital records.

Thanks to Mr. Walker’s comment, I’ve been keeping an eye out for the “margin notes” in digital systems. They’re photos of white board notes captured after a pivotal meeting and saved in a project folder, they’re comments in Google docs, they’re in the conversations that happen within project management tools like Asana or Basecamp.

How do I, with my records management hat on, ensure that these margin notes are represented without opening the flood gates to a plethora of non-records or duplication? How do I, with my archives hat on, ensure that the narrow stream of records preserved in the archives includes critical commentary and strategic thinking by key staff?

Knowing where to look for these digital margin notes, means that I must not only understand the kind of work my clients do, but also how they go about doing it. It also requires a generous definition of what constitutes a record with archival value.

Archives*RM Testimonial #5

This testimonial about the intersections of archives and records management comes from Elizabeth McGorty, Archivist & Records Manager for Brooklyn Navy Yard Development Corporation.

I serve as Archivist & Records Manager for the Brooklyn Navy Yard Development Corporation (BNYDC) which manages the Brooklyn Navy Yard. The Yard was once a federal ship repair facility on the Brooklyn waterfront owned and operated by the US Navy. During WWII it was the largest industrial complex in New York state – a 300-acre facility employing 70,000 civilian workers repairing over 1,000 ships. In 1967 the site was purchased by the City of New York and was transformed into an industrial park. There are now over 300 tenant businesses in the sectors of manufacturing, design, and art.

I like to think of the Yard as a site with two lives, and this is certainly present in our special collections – comprising personal effects of former welders, carpenters, and electricians, and our flagship collection(s) which comprise architectural and engineering drawings of the site – nearly 3,000 linear feet in extent. The collection of drawings created by the Bureau of Yards and Docks (construction arm of the US Navy) date as early as 1806, when military residences were built, but also contain drawings of substations, railroad tracks, shops, and production utility buildings. The other collection of drawings is contemporary, all commissioned by BNYDC for maintenance repair, renovation, and development projects, dating from the 1970s.

Architectural drawings are considered both a corporate record (as defined by our policies) and material of archival value. Thus, it requires arrangement, description, digitization, and ingest into our DAM/public facing digital library at the item level. Item level processing was predicated on user value – recalling boxes of project records currently in retention does not serve staff, nor the needs of external consultants and contractors who need specific drawings for repair and tenant fit-out projects that occur after the construction project is completed. Project records are a series in the schedule, but the drawings from those projects are accessioned into the two aforementioned collections.

Managing our corporate records (two types with the same name: corporate records deemed permanent, AND records of the corporation in temporary retention) has helped me as archivist committed to, among other things, preserving BNYDC’s corporate legacy. It’s easy to see the intersection of RM and Archives because they are both functions of a singular process: the life cycle of information, and I see this in practice every day.

Archivists and Records Managers, part 8

In 2005, two people working in Records and Archives at the World Health Organisation — Ineke Deserno and Donna Kynaston — had this to say about the intersection of records management and archival work:

“A records management program is indispensable for an archives program.  It ensures

  • the identification of records of long-term historical value and their orderly transfer to the archives
  • the regular, orderly elimination of large amounts of records that have no long-term value beyond their administrative usefulness
  • efficiency and economy in the management of the archives program by facilitating planning regarding space, records description, and records preservation”

They go on to explain,

“Effective records management programs ensure that records of permanent value that are transferred to the custody of an archives program in a regular, orderly fashion will be more readily accessible for reference use and will provide more reliable information for future users.  An effective records management program also ensures that records of no enduring value are not transferred to the archives simply as a means of disposing of them.”

[from “A Records Management Program that Works for Archives,” Information Management Journal (May/June 2005): 60-62]