Rank and frequency in Portuguese

AC/DC project, Linguateca

Em português

We give access here to the ferquency lists of tokens of the AC/DC corpora and of the other corpora -- specifically Web collections -- also made available by Linguateca.

The frequency lists of tha AC/DC corpora were created by the cwb-lexdecode tool from Open CWB / IMS-CWB from Stuttgart University. Lemma and part of speech were assigned in context by PALAVRAS, the Portuguese parser by Eckhard Bick (Bick, 2000). Be warned that the lists were computed from the automatically annotated versions of the corpora, most of them have not been revised.

Frequency and rank of wordforms and lemmas in the AC/DC corpora

The following service allows one to obtain frequencies and rank of lexical items and sublexical patterns, per corpus or all together:

Some comments to the choices taken:

CorpusToken frequency listLemma frequency list
All corpora 65,6 Mb 39,5 Mb
All corpora from Portugal 62,0 Mb 29,1 Mb
All corpora from Brazil 7,1 Mb 12,8 Mb
AmostRA-NILC 280 kb 120 kb
ANCIB 1,3 Mb 1,2 Mb
Avante! 2,1 Mb 1,4 Mb
CD HAREM 511 kb 263 kb
CETEMPúblico 72,7 Mb 24,7 Mb
CHAVE 12,2 Mb 16,1 Mb
CONDIVport 2,6 Mb 1,5 Mb
CoNE 788 kb 570 kb
DiaCLAV 1,8 Mb 1,7 Mb
ECI-EBR 1022 kb 431 kb
ECI-EE 71 kb 30 kb
ENPCPUB (parte em português) 211 kb 83 kb
Floresta 3,0 Mb 2,5 Mb
FrasesPB 98 kb 44 kb
FrasesPP 83 kb 38 kb
Museu da Pessoa 352 kb 139 kb
Natura/Minho 1,1 Mb 925 kb
NILC/São Carlos 7,0 Mb 7,3 Mb
todos juntos 25,5 Mb 38,6 Mb
Vercial 6,1 Mb 2,8 Mb

Partial searches, for few tokens or lemmata:

Tokens Lemmata

  • You can search using regular expressions of Perl.
  • To look for multiword lemmas, use a syntax like Belo=Horizonte, Castelo=Branco.

    Token frequencies of Web collections

    Depending on the collection, different methods were used:
    • For WBR99, we used the tokenization provided by the collection
    • For WPT03, another tokenization was used
    • For WPT05, the tokenization was done by the tokeniza function from the Lingua::PT::PLNbase library back in March 2009.

    we have not in any case attempted to remove foreign words from the lists.

    ColectionNo. documentsNo. wordsFrequency list disregarding capitalizationNo. different types disregarding capitalizationFrequency list keeping original capitalization No. different types keeping capitalization
    WPT-05 9.501.202 5.856.585.035 187M (gz) 25.237.118 206M (gz) 27.861.391
    WPT-03 1.529.758 1.059.436.086 55,1M (tar.gz) 6.834.451
    WBR-99 5.939.061 1.915.526.098 14M (tar.gz) 2.669.965

    Frequency lists of wordforms and lemmas by part of speech in the AC/DC corpora

    CorpusTokensLemmas
    NADJADVVNUMGRAMallNADJADVVNUMPROPGRAMallall/pos
    AmostRA-NILC 67 kb 30 kb 4 kb 64 kb 1 kb 5 kb 280 kb 52 kb 21 kb 3 kb 20 kb 2 kb 19 kb 1 kb 120 kb 134 kb
    ANCIB 348 kb 101 kb 13 kb 197 kb 57 kb 9 kb 1,3 Mb 265 kb 56 kb 10 kb 43 kb 59 kb 778 kb 3 kb 1,2 Mb 1,3 Mb
    Avante! 384 kb 232 kb 31 kb 572 kb 41 kb 12 kb 2,1 Mb 243 kb 114 kb 24 kb 65 kb 49 kb 956 kb 3 kb 1,4 Mb 1,5 Mb
    CD HAREM 111 kb 48 kb 7 kb 97 kb 7 kb 7 kb 511 kb 77 kb 30 kb 5 kb 23 kb 7 kb 119 kb 2 kb 263 kb 291 kb
    CETEMPúblico 2,5 Mb 1,2 Mb 109 kb 2,8 Mb 1014 kb 49,2 Mb 72,7 Mb 1,9 Mb 688 kb 92 kb 334 kb 1,1 Mb 20,7 Mb 8 kb 24,7 Mb 26,2 Mb
    CHAVE 2,0 Mb 976 kb 88 kb 2,2 Mb 827 kb 37 kb 12,2 Mb 1,4 Mb 532 kb 67 kb 256 kb 884 kb 13,0 Mb 7 kb 16,1 Mb 17,1 Mb
    CONDIVport 481 kb 299 kb 36 kb 648 kb 49 kb 24 kb 2,6 Mb 317 kb 160 kb 28 kb 80 kb 58 kb 898 kb 3 kb 1,5 Mb 1,6 Mb
    CoNE 189 kb 56 kb 7 kb 97 kb 38 kb 47 kb 788 kb 135 kb 30 kb 5 kb 24 kb 40 kb 334 kb 2 kb 570 kb 615 kb
    DiaCLAV 356 kb 189 kb 21 kb 478 kb 44 kb 11 kb 1,8 Mb 222 kb 93 kb 16 kb 57 kb 49 kb 1,2 Mb 3 kb 1,7 Mb 1,8 Mb
    ECI-EBR 218 kb 119 kb 16 kb 272 kb 5 kb 9 kb 1022 kb 151 kb 68 kb 12 kb 45 kb 8 kb 145 kb 2 kb 431 kb 478 kb
    ECI-EE 15 kb 10 kb 2 kb 14 kb 1 kb 3 kb 71 kb 11 kb 6 kb 2 kb 5 kb 1 kb 3 kb 1 kb 30 kb 34 kb
    ENPCPUB (parte em português) 45 kb 21 kb 5 kb 55 kb933 b 5 kb 211 kb 35 kb 14 kb 4 kb 15 kb 1 kb 10 kb 1 kb 83 kb 94 kb
    Floresta 631 kb 301 kb 33 kb 641 kb 56 kb 20 kb 3,0 Mb 428 kb 155 kb 28 kb 87 kb 59 kb 1,7 Mb 8 kb 2,5 Mb 2,7 Mb
    FrasesPB 27 kb 10 kb 2 kb 18 kb442 b 3 kb 98 kb 22 kb 8 kb 2 kb 8 kb407 b 2 kb 1 kb 44 kb 50 kb
    FrasesPP 21 kb 9 kb 2 kb 15 kb499 b 3 kb 83 kb 17 kb 7 kb 2 kb 6 kb467 b 2 kb 1 kb 38 kb 42 kb
    Museu da Pessoa 80 kb 27 kb 5 kb 102 kb 2 kb 7 kb 352 kb 57 kb 17 kb 3 kb 19 kb 2 kb 37 kb 2 kb 139 kb 156 kb
    Natura/Minho 231 kb 121 kb 14 kb 254 kb 34 kb 9 kb 1,1 Mb 148 kb 65 kb 11 kb 42 kb 38 kb 621 kb 2 kb 925 kb 998 kb
    NILC/São Carlos 1,2 Mb 589 kb 55 kb 1,3 Mb 567 kb 24 kb 7,0 Mb 804 kb 314 kb 43 kb 173 kb 598 kb 5,5 Mb 5 kb 7,3 Mb 7,9 Mb
    todos juntos 4,5 Mb 2,3 Mb 164 kb 6,5 Mb 1,7 Mb 141 kb 25,5 Mb 3,3 Mb 1,1 Mb 138 kb 757 kb 1,8 Mb 31,8 Mb 16 kb 38,6 Mb 41,4 Mb
    Vercial 1023 kb 504 kb 60 kb 2,0 Mb 27 kb 351 kb 6,1 Mb 709 kb 258 kb 49 kb 223 kb 32 kb 1,6 Mb 5 kb 2,8 Mb 3,1 Mb
    all 4,4 Mb 1,9 Mb 160 kb 4,9 Mb 1,7 Mb 49,6 Mb 65,6 Mb 3,3 Mb 1,1 Mb 133 kb 621 kb 1,9 Mb 32,6 Mb 13 kb 39,5 Mb 53,7 Mb
    all/pt 3,4 Mb 1,6 Mb 141 kb 4,4 Mb 1,1 Mb 49,6 Mb 65,6 Mb 2,5 Mb 917 kb 117 kb 529 kb 1,2 Mb 24,0 Mb 10 kb 29,1 Mb 39,6 Mb
    all/br 1,9 Mb 859 kb 76 kb 1,9 Mb 827 kb 36 kb 7,1 Mb 1,3 Mb 479 kb 59 kb 247 kb 874 kb 9,8 Mb 8 kb 12,8 Mb 17,6 Mb

    All tokens not belonging to any of the other categories were classified as GRAM (grammatical words). For that reason, these lists include an additional column specifying the category assigned by the parser.

    Similar information can be obtained for the Portuguese parts of COMPARA and CorTrad:

    CorpusTokensLemmas
    NADJADVVallNADJADVVProper namesall
    COMPARA (Portuguese) 545 kb 276 kb 42 kb 774 kb 1,7 Mb 327 kb 141 kb 37 kb 202 kb 219 kb 937 kb
    CorTrad jorn (Portuguese) 177 kb 104 kb 10 kb 190 kb 805 kb 122 kb 54 kb 7 kb 32 kb 235 kb 464 kb
    CorTrad literary (Portuguese) 39 kb 19 kb 4 kb 54 kb 186 kb 31 kb 12 kb 3 kb 15 kb 9 kb 73 kb
    CorTrad culinary (Portuguese) 30 kb 15 kb 1 kb 31 kb 120 kb 23 kb 8 kb 1 kb 10 kb 4 kb 48 kb

    Further information (so far only in Portuguese):

    [ Examples | Tokenization | Annotation | Corpora | Acknowledgements ]

    Last update: 26 Junho 2010.
    We would like to receive your feedback:
    Comments, requests and suggestions