Sebastian Aanstad | Kiara

Data collection for Hurtigruten Museum | Gaia Vesterålen 🛳️

Revive the Past: Innovative Digital Restoration for Hurtigrutemuseumet

Explore how cutting-edge technology can bring lost historical data back to life! Our latest project at sebastianaanstad.com showcases a remarkable digital restoration initiative for the Hurtigrutemuseumet in Sortland, utilizing a blend of TheWebArchive, automated web scraping with Puppeteer, and advanced data processing techniques. Keywords: Digital Restoration, Historical Data Recovery, Web Scraping Technology, Data Processing, Cultural Heritage Preservation

Cover image
Duration
2023 - 2024
Client
Museum Nord
Tags
#data
#website-scraping

This project details the innovative approach taken to recover valuable historical content from the Hurtigrutemuseumet’s lost website. Utilizing TheWebArchive, Puppeteer for scraping, and advanced data processing techniques, we successfully reconstructed the website’s archived data into a manageable, modern format.

Objectives:

  • To retrieve all available content from the archived versions of the Hurtigrutemuseumet website.
  • To convert the recovered HTML files into more accessible PDF and PNG formats.
  • To eliminate redundant data and enhance the quality of the recovered information.

Process

Detail the methods and technologies used throughout the project, emphasizing the technical expertise and tools.

Example:

  1. Data Retrieval: Accessed TheWebArchive to locate and retrieve historical snapshots of the Hurtigrutemuseumet’s website.
  2. Web Scraping: Employed Puppeteer to automate the scraping of HTML files from the archived site.
  3. Conversion to PDFs: Transformed each HTML file into PDF format for better usability and archiving.
  4. Image Conversion: Converted PDF files into PNG images for easier viewing and processing.
  5. Data Cleaning: Removed duplicate headers and footers from PNGs to ensure data cleanliness.
  6. Text Extraction: Extracted and compiled all textual content from the images, creating a searchable text database.

Challenges Faced

Discuss specific challenges encountered and how they were addressed, showcasing your problem-solving abilities.

Example:

  • Data Consistency: Ensured consistent formatting across different file types and recovery stages.
  • Automating File Conversion: Developed scripts to automate the conversion of numerous files, saving time and reducing manual errors.

Results

Highlight the outcomes and benefits realized by the museum from this project.

Example: The project culminated in a comprehensive digital archive of the museum's past online presence, providing the Hurtigrutemuseumet with a restored and searchable repository of historical data. This not only preserved important cultural heritage but also made it accessible for future educational and promotional uses.

Technologies Used:

  • TheWebArchive
  • Puppeteer (Node.js library for headless browser actions)
  • Adobe Acrobat (for PDF conversions)
  • Image processing scripts (for converting PDFs to PNGs and removing duplicates)
  • Optical Character Recognition (OCR) technology (for text extraction from PNGs)

Conclusion:

Reflect on the significance of the project, the technological insights gained, and potential further developments.

Example: This project underscores the critical role of digital preservation and the power of web archiving and scraping technologies in historical data recovery. Future enhancements could include implementing machine learning to automate the identification and removal of redundant elements more efficiently.