UF Digital Collections, Digital Library of the Caribbean, and Florida Digital Newspaper Library, FAQ

Production system: original-ufdc.uflib.ufl.edu

UF Digital Collections (UFDC), Digital Library of the Caribbean (dLOC), and Florida Digital Newspaper Library

The UF Digital Collections (UFDC), Digital Library of the Caribbean (dLOC), and Florida Digital Newspaper Library are undergoing major renovations 2022-2024.

The new sites are all live. The former site is still available at this time (planned retirement 2024 or prior): original-ufdc.uflib.ufl.edu

Please contact us: LTS-UFDC@uflib.ufl.edu.

User FAQ

  • I have a user who wants the old site!
    1. Have them use the original site: original-ufdc.uflib.ufl.edu.
    2. For items, just add /BIB/VID to the end and they will be at the items in Sobek. Example: https://original-ufdc.uflib.ufl.edu/AA00010090/00829
  • I found an item in the old site, but it doesn’t work in the new?
    • Please report the item to LTS-UFDC@uflib.ufl.edu and we will fix.
    • Reason: The old site has many items with invalid internal metadata. The new site enforces proper metadata, which ensures consistency of operations, but it means the invalid items do not show. Generally, we can resave these, and they will be fixed within an hour. We are batch processing through all items for this resave, but we have over 100,000 to go. Please report items so we can prioritize items that are being used.
  • What are the biggest changes with the new sites?
    1. The new sites are mobile responsive! The sites are easier for users overall, and vastly improved for usability and accessibility.
    2. The Florida Digital Newspaper Library has its first-ever portal! The new portal will be: https://newspapers.uflib.ufl.edu
    3. Users can search within titles! This is the most requested feature and new with the new system!
    4. User can search by date! This was another highly requested item and is now in place.
    5. Users can download the large Jp2 image files easily (see below for more on JP2 files)! Every item has a download link, and folks can directly download images.
    6. AskA is the contact, and it is easy for users to use!
    7. The first page users see is the citation page (example). This was highly requested, with users confused when seeing page images without descriptive metadata as the first thing.
    8. Audio and video files with proper transcripts in WebVTT now have a transcript text viewer along with the media file! This was another top requested item!
    9. The Library User Resources Discovery Committee is working on enhancements, and new documentation!
    10. New OAI and RSS feeds.
    11. Other than permanent links, links have changed. If you think you have a permanent link that is not working – what’s happening?
      1. The new system continues to support collection codes in the form of https://ufdc.ufl.edu/CODE
        1. If this isn’t working, please click on the “Collections & Partners” and go to the collection there. If you can’t find the collection, it may be one that has been removed in the clean-up process. Please use the main search to find an item, and then see what collections that item (and any others from the old collection) are now in.
      2. UFDC permanent links are only the items in this format: https://ufdc.ufl.edu/BIBID/VID
        1. If you have extra text (often a /l/ before BIBID or additional at the end like /allvolumes), you can edit the URL to the permanent link format to have it work.
    12. Collection pages are written for enduring relevance. Nearly every collection page is rewritten – please let us know if we missed anything or have any typos!
    13. Why do I only get 10,000 results (and it says 10,000+ above) when I can see there are more items based on my search/browse? The search (ElasticSearch) is set to deliver 10k results, to ensure searches return fast. While this can be programmatically changed to have more, it would mean searches could take 7-8 seconds, depending. We will be tuning this over 2022-2023. We recommend using the power of the search and modifying searches to get better results, fast, and to rely on the many search options now supported.
    14. When I do a search for something specific, like a full title, I got the item I wanted as the first result, and then a lot more items that have only a word in them (or a middle initial/letter)? This is how ElasticSearch is currently set–to give the most precise results first, and then to give any items that might be relevant (even if very low relevancy ranking).
      1. This is different than how Sobek-UFDC and many library systems are set, where they give fewer results, and this is closer to how many web search systems work. This is set for relevancy ranking, so a single letter typo will often still return the correct item/s.
      2. This will be investigated for what is optimal for users in 2022 with the usability work for the Florida Digital Newspaper Library’s ARPA grant.
    15. Why are the collection counts on the collection pages different from when I search?
      1. The search (ElasticSearch) is set to deliver 10k results, to ensure searches return fast. To know the correct number of what is (or should be) in a collection, use the counts from the collection landing page. Please note that updates are needed in the METS for aggregation codes, which results in discrepancies on the counts. These are being worked on.
    16. Some collections no longer exist. The Libraries completed the first-ever collection cleanup, removing unused and extraneous collections, going from over 900 to ~468. The collections are now more useful and usable for users, but this is nearly a 50% change and work remains.
  • What is the system called? The production system is Sobek (or SobekCM), so what’s the new?
    1. Please refer to the portal: UF Digital Collections (UFDC), Digital LIibrary of the Caribbean (dLOC), and Florida Digital Newspaper Library (FDNL). Or, please refer to UFDC and all digital collections hosted at UF.
    2. For the technologies, we are working with component parts and not a monolith, so the system is best referred to by the components: Linux, PostgreSQL, Python, React, APIs, EslasticSearch, etc.  Please contact LTS for any specific language needs: LTS-UFDC@uflib.ufl.edu.
  • Why are there some items without thumbnails in search?
    1. It depends.  Please report these to us, so we can check, and so that we have more information to collect for routing for corrections.
    2. To date, reasons include: 1) there are no thumbnails; and, 2) the item has thumbnails, but is wrong in the metadata (missing mainthumbnail in the procparam in the METS, which can be fixed as a metadata update).
  • LTS is the contact for any system questions, providing full support, just like for other web systems in the Libraries.  For any questions, contact LTS-UFDC@uflib.ufl.edu.
  • What are the biggest technical changes?
    • The biggest and best changes are under the hood and in more mature technical practices. The new system utilizes modern development practices. There are changes for all development aspects: proper development routing (test for development, staging for review, and live – with three systems/processes needed to ensure things work), proper processes for deployment, test coverage (meaning when things are fixed, they are tested and automatically validated for future, so there is no more having the same bug), and more that mean we can expect higher up time, fewer problems, and a more solid and maintainable system overall. Additionally, we are utilizing a PM/PO (project manager and product owner) model for defining and prioritizing needs, and working with the Library User Resources Discovery Committee as the stakeholder representatives for the patron sites.
    • We confirmed using METS as the source of truth (as opposed to mix of METS and DB), METS contents are more visible, and there are many data issues that can now be corrected, and which were confusing on the old site.
  • Searching in an item?
    • Can search terms be highlighted on the page image? This is pending additional work, which are queued for 2022-2023.
    • When I search, do I have to search again once I get to the item? Yes, for the beta release. We are looking at passing through the searches and display options for that, which are queued for 2022-2023.
  • Terms:
    • Title sets: refer to publications that were published in successive order. This collection provides access to newspapers, journals, magazines, newsletters, and other serial publications.
    • Genre: in library terminology, genre means the form or type of material. Clicking on genre brings up the list of all available, including newspapers, serials, audio, birth records, and nearly 200 more.
    • Jp2 files: JP2 is a type of image file that has higher resolution, and this is what enables the zoomable image views. For more information on Information on free JP2 viewers, see the Florida Department of Environmental Protection and the Irfanview software is available as freeware to view, convert, and batch convert JP2 images.
  • Citing items, bookmarks and saved searches: What are the recommended tools?
    • Use Zotero for saving items and citations, and for citation management.
    • Use Hypothes.is for annotating and noting pages.
  • Searching for text in an item and nothing shows?
    • In October 2021, we discovered thousands of items with text files that are blank (example). These are the same as in the original UFDC. We are currently investigating the problem. Once scope is known, we will look to a solution.
  • How do I search for Aerials?
    • If using the map search, please note that the coordinates are estimated based on a calculation and are not exact. Also, only about half have coordinates.
  • Statistics: what does a view mean?
  • Downloads: Can I bulk download files for an API or a collection?
  • API: How do I use the API or what other data access points are there?

Technology

  • How is the new system built?
    • The new UFDC uses standard technologies and practices in use by other webscale systems as with UNT, Europeana, NARA, Getty, and others. Technologies include: Python, APIs, ElasticSearch, ReactJS, PostgreSQL and more. The new UFDC has separate production and patron databases and operations. Technology documentation will be updated in 2022-2023 as part of TRAC.