Viewing profile — anas-awadalla
anas-awadalla
HN member- Joined
- Tue, Aug 04, 2020, 6:14 PM UTC
- HN karma
- 15
- Public activity
- 2 items
- HN profile
- View on Hacker News ↗
About anas-awadalla
No profile information was provided.
Recent public activity
-
comment
Comment #41064146
Hello! Totally agree that tokens will be model dependent. We chose to calculate tokens using the GPT-2 tokenizer as that is a common metric used by other datasets like fineweb. So …
-
comment
Comment #41064131
Hello! Creator of MINT here. We do a lot of pre-processing of commoncrawl (which in its raw form isn’t all that useful for training models). This includes heuristics to remove low …