Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: GEITje-7B – A New Large Open Dutch Language Model

github.com

Post reply on HN
Print view View original

1–5 of 5 posts

Show HN: GEITje-7B – A New Large Open Dutch Language Model

#1

Post by erijgersberg »

Tried my hand at training a large open Dutch language model, for which the pickings are slim right now.

GEITje is a large open Dutch language model with 7 billion parameters, based on Mistral 7B. I've continued pretraining it on 10 billion tokens of Dutch text. This has improved its Dutch language skills and increased its knowledge of Dutch topics. There's an experimental chat-finetuned model too, called GEITje-chat.

I have to say the experience of gathering a dataset and training a model has been very educational for me. Being forced to deal with every detail yourself really deepens your understanding of a subject.

It's all in the hands of the community now. Can't wait to see what they do with it!

Want to try it out? There's a demo live at Hugging Face Spaces right now: https://huggingface.co/spaces/Rijgersberg/GEITje-7B-chat

Show HN: GEITje-7B – A New Large Open Dutch Language Model
github.com

Re: Show HN: GEITje-7B – A New Large Open Dutch Language Model

#3

Post by erijgersberg »

vermaat wrote: post #2

Works great, does indeed have more Dutch knowledge besides only speaking Dutch.

It sure does! There's still lots of room for improvement though. It still has trouble sometimes with the industry-standard Bassie en Adriaan-test.

It's too bad that there is still a distinct lack of suitable Dutch training sets and benchmarks.

Re: Show HN: GEITje-7B – A New Large Open Dutch Language Model

#4

Post by vermaat »

erijgersberg wrote: post #3
vermaat wrote: post #2

Works great, does indeed have more Dutch knowledge besides only speaking Dutch.

It sure does! There's still lots of room for improvement though. It still has trouble sometimes with the industry-standard Bassie en Adriaan-test. It's too bad that there is still a distinct lack of suitable Dutch training sets and benchmarks.

I hope the 13,5 million Euro funds will be used wisely for the GPT-NL project. It would be nice if they open-source it (or at least the training dataset).
Post reply on HN