| title: README | |
| emoji: 📉 | |
| colorFrom: green | |
| colorTo: yellow | |
| sdk: static | |
| pinned: false | |
| # German Wikipedia LMs (GWLMs) | |
| We present Language Models ([BERT](https://huggingface.co/gwlms/bert-base-dewiki-v1), [BERT with Token Dropping](https://huggingface.co/gwlms/bert-base-token-dropping-dewiki-v1), [TEAMS](https://huggingface.co/gwlms/teams-base-dewiki-v1-discriminator), [T5](https://huggingface.co/gwlms/t5-efficient-large-dewiki-v1)) pretrained on German Wikipedia. | |
| This is an ongoing project! | |
| # German Wikipedia Corpus | |
| We use a recent Wikipedia Dump, that can can be accessed [here](https://huggingface.co/datasets/gwlms/dewiki-20230701). Additionally, a sentence-segmented (using NLTK) is available [here](https://huggingface.co/datasets/gwlms/dewiki-20230701-nltk-corpus). | |
| # Fine-tuned Models | |
| We fine-tuned NER models using [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) library on GermEval 2014 NER dataset and upload the best models: | |
| * [GermEval 2014 NER Model with BERT as backbone LM](https://huggingface.co/gwlms/span-marker-bert-germeval14) | |
| * [GermEval 2014 NER with BERT + Token Dropping as backbone LM](https://huggingface.co/gwlms/span-marker-token-dropping-bert-germeval14) | |
| * [GermEval 2014 NER Model with TEAMS as backbone LM](https://huggingface.co/gwlms/span-marker-teams-germeval14) | |
| # Acknowledgements | |
| Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC). | |
| Many Thanks for providing access to the TPUs ❤️ |