Technology

40278 readers

255 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 3 years ago

MODERATORS

alyaza@beehaw.org

TheRtRevKaiser@beehaw.org

gyrfalcon@beehaw.org

rs5th@beehaw.org

coldredlight@beehaw.org

SemioticStandard@beehaw.org

TheRtRevKaiser@kbin.social

remington@beehaw.org

Why extracting data from PDFs is still a nightmare for data experts (arstechnica.com)

submitted 6 months ago by misk@sopuli.xyz to c/technology@beehaw.org

7 comments fedilink hide all child comments

Countless digital documents hold valuable info, and the AI industry is attempting to set it free.

For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files. These digital documents serve as containers for everything from scientific research to government records, but their rigid formats often trap the data inside, making it difficult for machines to read and analyze.

"Part of the problem is that PDFs are a creature of a time when print layout was a big influence on publishing software, and PDFs are more of a 'print' product than a digital one," Derek Willis, a lecturer in Data and Computational Journalism at the University of Maryland, wrote in an email to Ars Technica. "The main issue is that many PDFs are simply pictures of information, which means you need Optical Character Recognition software to turn those pictures into data, especially when the original is old or includes handwriting."

you are viewing a single comment's thread
view the rest of the comments

[–] adespoton@lemmy.ca 25 points 6 months ago (3 children)

This is silly.

PDF is a Portable Document Format. It replaced Encapsulated Postscript as a document storage medium. It’s not all that different than a more highly structured zip archive, designed to structure how the layout and metadata is stored as well as the content.

It has a spec, and that spec includes accessibility features.

The problem is that how many people use it is to take a bunch of images of varying quality and place them on virtual letter-sized pages, sometimes interspersed with form fields and scripts.

A properly formatted accessible PDF is possible these days with tools available on any computer; these are compact, human and machine readable, and rich in searchable metadata.

Complaining about inaccessible PDFs is sort of like complaining about those people who use Excel as a word processor.

So, with that out of the way… on to the sales pitch: “use AI to free the data!”

Well I’m sorry, but most PDF distillers since the 90s have come with OCR software that can extract text from the images and store it in a way that preserves the layout AND the meaning. All that the modern AI is doing is accomplishing old tasks in new ways with the latest buzzwords.

Remember when PDFs moved “to the cloud?” Or to mobile devices? Remember when someone figured out how to embed blockchain data in them? Use them as NFTs? When they became “web enabled?” “Flash enabled?”

PDF, as a container file, has ridden all the tech trends and kept going as the convenient place to stuff data of different formats that had to display the same way everywhere.

It will likely still be around long after the AI hype is long gone.

[–] FaceDeer@fedia.io 10 points 6 months ago

This is silly.

Whether it's "silly" or not is irrelevant, the problem described in the article is real. I have seen innumerable PDFs over the years that were atrocious when it came to the use of those accessibility features, the format's design factors in to how people use it and people use it terribly. If plain old OCR were enough then this wouldn't be such a problem.

load more comments (2 replies)