Accuracy, reported openly
What matters most is how much personal data remains after masking. We therefore report both exact-span F1 and complete misses, with the test texts, method and limitations open for review.
Misses are the most important metric
A complete miss means that a piece of personal data remains in plain text. Precision shows how often a detection was correct, while recall shows what proportion of all personal data was found. Exact-span F1 combines the two. Complete misses are reported separately.
Current test results
The published model was measured on synthetic Swedish test sets on 2026-08-06. The results show the model's known strengths and limits.
- mixed Swedish test texts: F1 / misses
- 96.9% / 1 of 205
- synthetic addresses: F1 / misses
- 100.0% / 0 of 57
Compared with KBLab
Maskera is a small Swedish NER model built for local masking. KBLab lowermix is a full-size Swedish BERT model whose casing mix makes it more robust on text without capitals. Maskera's published 43 MB q4 model and KBLab lowermix's 496 MB fp32 model were run on 2026-08-11 over the same 121 synthetic Swedish texts with 211 names, places and organisations. An entity counts as masked when a detection overlaps it; typed F1 additionally requires the correct category.
Original casing
| System | Masked | Typed F1 |
|---|---|---|
| Maskera v19 q4 | 211 of 211 (100.0%) | 87.1% |
| KBLab lowermix fp32 | 205 of 211 (97.2%) | 89.4% |
The same texts lowercased
| System | Masked | Typed F1 |
|---|---|---|
| Maskera v19 q4 | 211 of 211 (100.0%) | 85.7% |
| KBLab lowermix fp32 | 187 of 211 (88.6%) | 83.2% |
Maskera masked all 211 labelled names, places and organisations in both runs. KBLab had the higher typed F1 with original casing; Maskera had the higher typed F1 in lowercase.
Maskera's developer wrote the test, so it shows a direction rather than an independent ranking. Street addresses and Maskera's post-processing are excluded.
End-to-end test with LogosGuard
LogosGuard is a Chrome extension for masking in the browser, not a standalone NER model. This therefore compares LogosGuard's complete file-masking flow with Maskera's complete local pipeline. Maskera v19 and LogosGuard 2.4.4 were run on 2026-08-14 over the same 258 synthetic Swedish texts with 952 labelled pieces of personal data, including names, addresses, Swedish personal and coordination numbers, phone numbers, and government, healthcare and legal language. LogosGuard ran on the free plan with Balanced accuracy. Both systems used the same strict measure: a full hit requires every letter and digit from the labelled value to be removed.
| System | Fully masked | Partial leaks | Plain-text misses |
|---|---|---|---|
| Maskera v19 q4 | 933 of 952 (98.0%) | 8 | 11 |
| LogosGuard 2.4.4 | 606 of 952 (63.7%) | 49 | 297 |
Maskera fully masked 327 more of the 952 labelled values. Some material remained in 19 cases with Maskera and 346 cases with LogosGuard — a 34.3 percentage-point difference in full-hit rate.
Maskera's developer wrote the synthetic test set, so this is author-coupled evidence rather than an independent ranking. The corpus is not exhaustively annotated for precision. LogosGuard's file export also converted Swedish UTF-8 characters into Windows-1252 mojibake; the characters were restored before scoring.
Broader historical NER comparison
The repository's broader model run covers eleven rows on the same 22 independent Swedish Wikipedia texts with 58 labelled people, places and organisations. Competitors were measured from 2026-07-04 through 2026-07-19, and Maskera's v18 row was measured on 2026-07-19. Overlap counts as masked; typed F1 additionally requires the correct category.
Original casing
| Model or library | Size | Masking recall | Typed F1 |
|---|---|---|---|
| Maskera studentSmall Swedish NER model for local masking | 43 MB | 97% | 96% |
| KBLab lowermix reallysimple-nerSwedish BERT model designed for casing robustness | ≈475 MB | 100% | 94% |
| nbailab scandi-nerMultilingual Scandinavian NER model | ≈500 MB | 100% | 94% |
| KBLab neriob (IOB head)Swedish BERT model with an IOB classification head | ≈475 MB | 100% | 94% |
| KB-NER (the SUC classic)Classic Swedish BERT model trained on SUC | ≈475 MB | 100% | 92% |
| KBLab reallysimple-nerCasing-sensitive Swedish BERT model for NER | ≈475 MB | 98% | 91% |
| RecordedFuture Swedish-NERSwedish NER model trained on internet and forum text | ≈500 MB | 100% | 88% |
| sbx PII general / detailedgeneraldetailedTwo KB-BERT models with a different PI label scheme | ≈475 MB | 5 / 10% | 10 / 19% |
| Rampart (nationaldesignstudio)Small multilingual browser PII model | 14.7 MB | 34% | 42% |
| Desert Ant redact (multilingual)Multilingual on-device redaction model | 13 MB | 36% | 53% |
| swedish-pii (regex + name lists)Rules and Swedish name lists, not an AI model | 2.5 MB | 50% | 52% |
The same texts lowercased
| Model or library | Masking recall | Typed F1 |
|---|---|---|
| KBLab lowermix reallysimple-ner | 97% | 90% |
| Maskera student | 95% | 91% |
| RecordedFuture Swedish-NER | 83% | 82% |
| nbailab scandi-ner | 69% | 77% |
| KB-NER (the SUC classic) | 28% | 35% |
| KBLab reallysimple-ner | 29% | 39% |
| KBLab neriob (IOB head) | 24% | 33% |
| Desert Ant redact (multilingual) | 24% | 39% |
| sbx PII general / detailedgeneraldetailed | 9% | 16% |
| swedish-pii (regex + name lists) | 0% | 0% |
In the dated v18 run, Maskera had the highest typed F1 on originally cased text. On lowercase text, KBLab lowermix had the highest masking recall while Maskera had the highest typed F1.
These are historical v18 results and must not be read as v19 scores. The independent raw corpus was removed before the privacy-clean v19 release, so the aggregates are retained for transparency but cannot be reproduced document by document from today's checkout. Separate system tests including Presidio, Privacy Filter, EU PII Safeguard and Blindfold use different runs and are reported in BENCHMARKS.md.
How to interpret the results
No labelled data was left entirely in plain text in the synthetic address test. The mixed Swedish text test missed 1 of 205.
The only miss was a broad public geographic region, not a person, contact detail, identifier or street address.
All 35 street addresses were found with an exact span, including the house number.
What these tests do not show
- The test sets are synthetic or author-coupled. They are used to prevent regressions, not as an independent rating of all Swedish text.
- No automatic masking is perfect. Unusual names, OCR noise and phrasing far from the training data can be missed. Have a person review the text when an error could have serious consequences.
- These corpora measure free text. Structured identifiers such as Swedish personal identity numbers and IBANs are handled by the rule engine and are not included in these model tests.
Sources and reproduction
Published maskera-sv-ner v19 (q4, about 43 MB) was measured on the synthetic release gates on 2026-08-06.
The test texts, method, runner scripts and complete Maskera results are available in the open repository. BENCHMARKS.md is the source of truth if a number elsewhere differs.