SaaS Metrics
Hello HN, over the past 7 months I've spent nearly 3,000 hours on building SNEWPAPERS, the first historical newpaper archive with full-text extractions, nearly perfect OCR, a vast categorization taxonomy and of course with semantic and agentic search capabilities.Problem:
I wanted to search through newspaper archives, but when I tried every service only lets you search for keywords and dates, and gives you back raw images of the papers, and too many of them with no context. A sea of noise.Solution:
I taught machines how to read the newspapers and so far I've extracted the content from > 600k pages (about 5TB) from the Chronicling America collection. Problems I had to deal with were an infinite variety of layouts, font sizes, image scan qualities, resolutions, aspect ratios, navigating around the images on the page. I also had to figure out how to get OCR to be nearly perfect so people wouldn't hate reading the extracts. I stitched together a multi-model pipeline (layout tech, ocr tech, llm, vllm) with heuristics to go from layout -> segmentation -> classification. I put it all in OpenSearch / Postgres and made it semantically searchable and also put an agentic search tool on top that knows how to use the API really well and helps you write queries to find what you're looking for. Happy to discuss AWS architecture and scaling as well, that was tough!If you have five minutes and you just want to jump in and have your own personalized experience, what I would suggest is:Before searching for anything, go to the Sleuth page
Ask it about anything from 1736 to 1963, maybe 1 or 2 follow up questions
Then go to the search page so you can see the queries it wrote for you (bottom left "saved queries") and uncover more info on whatever it is you're interested inIf you think it's cool and you want to learn more, then there's about 10 minutes of video guides on the various capabilities in "Guide" on the nav barSome other people have also taken a crack at this, notably:https://dell-research-harvard.github.io/resources/americanst... (very good attempt)
https://labs.loc.gov/work/experiments/newspaper-navigator/ (focused on images)
Show HN: Large Scale Article Extract of Newspapers 1730s-1960s
The first historical newspaper archive offering full-text extraction, near-perfect OCR, and advanced semantic/agentic search, solving the problem of noise and lack of context in traditional keyword-based archives.
View Origin LinkProduct Positioning & Context
AI Executive Synthesis
The first historical newspaper archive offering full-text extraction, near-perfect OCR, and advanced semantic/agentic search, solving the problem of noise and lack of context in traditional keyword-based archives.
Traditional historical archives are severely limited by keyword-only search and raw image returns, creating significant research friction. SNEWPAPERS addresses this by applying advanced AI/ML techniques to transform unstructured historical data into semantically searchable, contextualized information. The achievement of "nearly perfect OCR" on diverse historical layouts is a critical technical hurdle overcome, enabling reliable full-text extraction. This product has profound implications for academic research, historical analysis, and potentially even legal or journalistic investigations, where accurate, contextualized access to historical documents is paramount. The integration of semantic and agentic search capabilities represents a significant leap in data discoverability, moving beyond simple retrieval to intelligent information synthesis. This demonstrates the power of AI to unlock value from previously inaccessible or unwieldy datasets.
Related Ecosystem & Alternatives
Discover adjacent products, open-source repositories, and developer tools sharing similar technical architecture.
Deep-Dive FAQs
What is Large Scale Article Extract of Newspapers 1730s-1960s?
Large Scale Article Extract of Newspapers 1730s-1960s is analyzed by our AI as: The first historical newspaper archive offering full-text extraction, near-perfect OCR, and advanced semantic/agentic search, solving the problem of noise and lack of context in traditional keyword-based archives.. It focuses on Traditional historical archives are severely limited by keyword-only search and raw image returns, creating significant research friction. SNEWPAPE...
Where did Large Scale Article Extract of Newspapers 1730s-1960s originate?
Data for Large Scale Article Extract of Newspapers 1730s-1960s was aggregated directly from the Hacker News community ecosystem, representing raw developer and early-adopter sentiment.
When was Large Scale Article Extract of Newspapers 1730s-1960s publicly launched?
The initial public indexing or launch date for Large Scale Article Extract of Newspapers 1730s-1960s within our tracked developer communities was recorded on May 2, 2026.
How popular is Large Scale Article Extract of Newspapers 1730s-1960s?
Large Scale Article Extract of Newspapers 1730s-1960s has achieved measurable traction, logging over 27 traction score and facilitating 14 recorded discussions or engagements.
Which technical categories define Large Scale Article Extract of Newspapers 1730s-1960s?
Based on metadata extraction, Large Scale Article Extract of Newspapers 1730s-1960s is categorized under topics such as: historical newspaper archive, full-text extractions, nearly perfect OCR, categorization taxonomy.
What are some commercial alternatives to Large Scale Article Extract of Newspapers 1730s-1960s?
Our semantic intelligence engine identifies potential commercial alternatives in the SaaS space, such as Bluedot 2.1, which offers overlapping value propositions.
How does the creator describe Large Scale Article Extract of Newspapers 1730s-1960s?
The original author or development team describes the product as follows: "Hello HN, over the past 7 months I've spent nearly 3,000 hours on building SNEWPAPERS, the first historical newpaper archive with full-text extractions, nearly perfect OCR, a vast categorization ta..."
Community Voice & Feedback
How well do you think your OCR solution would work on magazines? I found OCR very hit and miss with magazines, especially ones with text into background pictures etc.
I'm opening up https://snewpapers.com/today-in-history to the world right now. Per @benwillis's advice below, I will figure out how to make a section of the data searchable for free as well, but this is the best I can do today!Thank you everyone for taking the time to look at snewpapers today, enjoy!
[dead]
It would be really cool to create different analysis across the time- Each month's / year's top news headline- Left / Right swings of publishers
Looks cool, congrats!I've also worked with this data, but only for research purposes:https://www.finhist.com/bank-runs/episodes/13895.html
https://www.finhist.com/bank-runs/index.htmlSurprisingly, I found out that layout was the trickiest thing, as newspaper articles often had multiple layers of headers, spanned multiple columns, etc.Do you have a preferred solution on that?
https://www.finhist.com/bank-runs/index.htmlSurprisingly, I found out that layout was the trickiest thing, as newspaper articles often had multiple layers of headers, spanned multiple columns, etc.Do you have a preferred solution on that?
As someone who has done a lot of downloading/parsing, this is so awesome and impressive to see.One thing to think about, which I also struggle with when it comes to large and complicated datasets, is the UI. Even being in the search industry for a long time, it's difficult for me to concretely see how I would use this.I'd suggest taking a small sample of the dataset that might be reflective of how people would use it, then make that segment public and immediately searchable without registering. eg: One year of articles related to the Olympics.What I've found is that it's hard for a lot of people to imagine how they would use something without actually using it. So giving people the actual experience of searching the archive and interacting with the results would go a long way.Again, congrats on the work. This is really impressive work.
A few examples you can click on without having to authenticate or click the free trial (no cc if you do though and I won't bother you or chase you with spam etc...)https://snewpapers.com/components/b2d40c08-db63-40e8-890f-09...https://snewpapers.com/components/0fabc8e4-a60b-4f31-9ad1-b0...https://snewpapers.com/components/cdde790f-4e97-4f2d-a2c2-95...
Discovery Source
Hacker News Aggregated via automated community intelligence tracking.
Tech Stack Dependencies
No direct open-source NPM package mentions detected in the product documentation.
Media Tractions & Mentions
No mainstream media stories specifically mentioning this product name have been intercepted yet.
Deep Research & Science
No direct peer-reviewed scientific literature matched with this product's architecture.