FinePDFs: Liberating 3T of the finest tokens from PDFs
📄
18
FineData
community
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
View all activity
Papers
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
How to use this dataset to extract PDFs by subject?
👍
1
1
#14 opened 4 months ago
by
vgoklani
Can additional corpuses further train this model?
1
#13 opened 4 months ago
by
fenjamin
Decontamination against benchmarks?
4
#11 opened 4 months ago
by
jo-kn
MarCognity-AI for HuggingFaceFW/finepdfs
1
#23 opened 3 months ago
by
elly99
hynky
updated
a
Space
about 7 hours ago
hynky
published
a
Space
1 day ago
guipenedo
updated
a
Space
1 day ago
craffel
authored
a
paper
13 days ago
eliebak
submitted
a
paper
to Daily Papers
20 days ago
Which language detector did you use
4
#28 opened 24 days ago
by
ming030890
The "file_path" data field appears to primarily contain cc-index paths rather than WARC paths.
9
#16 opened 4 months ago
by
lnstrument
A Few Questions About the Implementation Details of the finepdfs Project
5
#24 opened 3 months ago
by
yoliax
Dataset broken by latest update?
5
#27 opened about 1 month ago
by
Rijgersberg
hynky
updated
a
dataset
about 1 month ago
Post
3874
🤖 Did you know your voice might be cloned without your consent from just *one sentence* of audio?
That's not great. So with @frimelle , we brainstormed a new idea for developers who want to curb malicious use: ✨The Voice Consent Gate.✨
Details, code, here: https://huggingface.co/blog/voice-consent-gate
That's not great. So with @frimelle , we brainstormed a new idea for developers who want to curb malicious use: ✨The Voice Consent Gate.✨
Details, code, here: https://huggingface.co/blog/voice-consent-gate
- 3 replies
thomwolf
authored
a
paper
3 months ago
lvwerra
authored
a
paper
3 months ago
Post
2911
🤖 As AI-generated content is shared in movies/TV/across the web, there's one simple low-hanging fruit 🍇 to help know what's real: Visible watermarks. With the Gradio team, I've made sure it's trivially easy to add this disclosure to images, video, chatbot text. See how: https://huggingface.co/blog/watermarking-with-gradio
Thanks to the code collab in particular from @abidlabs and Yuvraj Sharma.
Thanks to the code collab in particular from @abidlabs and Yuvraj Sharma.
davanstrien
posted
an
update
4 months ago
Post
1541
I fine-tuned a smol VLM to generate specialized art history metadata!
https://huggingface.co/davanstrien/iconclass-vlm: Qwen2.5-VL-3B trained using SFT to generate ICONCLASS codes (think Dewey Decimal for art!)
Trained with TRL + HF Jobs - single UV script, no GPU needed!
Space to explore predictions on a test set: davanstrien/iconclass-predictions
Blog soon!
https://huggingface.co/davanstrien/iconclass-vlm: Qwen2.5-VL-3B trained using SFT to generate ICONCLASS codes (think Dewey Decimal for art!)
Trained with TRL + HF Jobs - single UV script, no GPU needed!
Space to explore predictions on a test set: davanstrien/iconclass-predictions
Blog soon!
Post
3896
Super excited to announce that our research team at Hugging Face will be doing an AMA on reddit r/LocalLLaMA.
Come ask any questions to the team behind SmolLM, FineWeb and more! And who knows, maybe there’ll be a shiny new release to talk about?
Thursday 4th September, 8AM-11AM PST 🤗
science
Come ask any questions to the team behind SmolLM, FineWeb and more! And who knows, maybe there’ll be a shiny new release to talk about?
Thursday 4th September, 8AM-11AM PST 🤗