399 lines
56 KiB
Plaintext
399 lines
56 KiB
Plaintext
|
Ë
|
|||
|
|
?û gy¶ ã óÊ — d dl Z d dlZd dlZd dlZd dlmZmZ d dlZd dlm Z d dl
|
|||
|
|
m Z m Z d dl
|
|||
|
|
mZmZmZmZmZmZmZmZ d dlmZ d dlmZ d dlmZ d d l m Z d d
|
|||
|
|
l m Z m Z m!Z!m"Z" d d l#m$Z$ ejJ d
|
|||
|
|
„ « Z& ejJ « d„ « Z'ejJ d„ « Z(ejR jU d« d„ « Z+ejR jU d« d„ « Z,dZ-ejR jU d« d„ « Z.d„ Z/d„ Z0d„ Z1d„ Z2d„ Z3d „ Z4d „ Z5d „ Z6d „ Z7ejR jq d « d!„ « Z9d"„ Z:d#„ Z;d$„ Z<d%„ Z=d&„ Z>d'„ Z?d(„ Z@d)„ ZAd*„ ZBd+„ ZCd,„ ZDd-„ ZEd.„ ZFd/„ ZGd0„ ZHd1„ ZId2„ ZJejR j— d3g d4¢d5d6gd gd gd7ggd d7gd ggffg d8¢d9d6gd gd gd gd7ggg d:¢d;ggffd<d=gd5d6gd gd d7ggd d7gd7ggffg d>¢g d?¢d d7gd gd ggd gd gd7d ggffg d?¢g d@¢d gd7gd d;ggd gd7gd gd ggffdAd<gd<gg d ggd7ggffg dB¢g dC¢d gd d7gd7gd ggd d7gd7d gd;ggffg« dD„ « ZLdE„ ZMdF„ ZNdG„ ZOdH„ ZPdI„ ZQdJ„ ZRdK„ ZSdL„ ZTdM„ ZUdN„ ZVdO„ ZWdP„ ZXdQ„ ZYdR„ ZZdS„ Z[dT„ Z\y)Ué N)ÚAdamÚ compounding)ÚEnglish)ÚDocÚDocBin)Ú AlignmentÚCorpusÚExampleÚbiluo_tags_to_offsetsÚbiluo_tags_to_spansÚ docs_to_jsonÚ iob_to_biluoÚoffsets_to_biluo_tags©Úget_alignments)ÚAlignmentArray)Ú json_to_docs)Útrain_while_improving)Úget_words_and_spacesÚload_config_from_strÚload_model_from_pathÚ minibatché )Ú make_tempdirc óö — t « } g d¢}g d¢}g d¢}g d¢}g d¢}g d¢}g d¢}dgt |« z }d |d
|
|||
|
|
< d |d < d
|
|||
|
|
|d< d|d< d|d< dddœ} t | j ||||||||¬« }
|
|||
|
|
| |
|
|||
|
|
_ |
|
|||
|
|
S )N©
|
|||
|
|
ÚSarahú'sÚsisterÚflewÚtoÚSiliconÚValleyÚviaÚLondonú.)
|
|||
|
|
ÚNNPÚPOSÚNNÚVBDÚINr' r' r+ r' r& )
|
|||
|
|
ÚPROPNÚPARTÚNOUNÚVERBÚADPr, r, r0 r, ÚPUNCT)
|
|||
|
|
úNounType=prop|Number=singzPoss=yesz Number=singzTense=past|VerbForm=finÚ r2 r2 r3 r2 zPunctType=peri)
|
|||
|
|
r r é r4 r4 é é r4 é é )
|
|||
|
|
ÚpossÚcaseÚnsubjÚROOTÚprepÚcompoundÚpobjr= r? Úpunct)
|
|||
|
|
r r r Úflyr! r" r# r$ r% r& ÚOúB-PERSONr zI-PERSONé úB-LOCr8 úI-LOCr5 zB-GPEé ç ð?ç )ÚTRAVELÚBAKING)ÚwordsÚtagsÚposÚmorphsÚheadsÚdepsÚlemmasÚents)r Úlenr ÚvocabÚcats) ÚnlprL rM rN rO rP rQ rR rS rV Údocs úcC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/tests/training/test_training.pyrX rX # s® € ä
|
|||
|
|
‹)€Câ ^€EÚ L€DÚ
|
|||
|
|
]€Cò=€Fò
|
|||
|
|
+€EÚ a€DÚ
|
|||
|
|
^€FØ ˆ5”3�u“:Ñ €DØ€Dˆ�GØ€Dˆ�GØ€Dˆ�GØ€Dˆ�GØ€Dˆ�GØ SÑ )€Dä
|
|||
|
|
|
|||
|
|
Ø � ‰ ØØ
|
|||
|
|
Ø ØØØ
|
|||
|
|
ØØ
|
|||
|
|
ô
|
|||
|
|
€Cð €C„HØ €Jó c ó& — g d¢g d¢g d¢g d¢g d¢dœS )N)rD r r4 r6 r8 r5 r7 )ÚHiÚthereÚeveryoneÚItÚisÚjustÚme)TTTTTTF)ÚINTJÚADVÚPRONre ÚAUXrd re )rD r r rD r r r )ÚidsrL ÚspacesrM Ú sent_starts© rj rZ rY Ú merged_dictrk H s € ò %ÚFÚ=ÚEÚ,ñ ð rZ c ó. — t « } | j S ©N)r rU )rW s rY rU rU S s € ä
|
|||
|
|
‹)€CØ �9‰9ÐrZ iç c
|
|||
|
|
ó& — dg gdg gdg gdg gdg gdg gddggd d
|
|||
|
|
ggd d ggg } t « }|j d
|
|||
|
|
« }| D ]! \ }}|D ] \ }}}|j |« Œ Œ# |j « t d« D ]Z }t j | « | D ]> \ } }
|
|||
|
|
t j |j | « d|
|
|||
|
|
i« } |j | g« Œ@ Œ\ t « 5 } |j | « t | « }
|
|||
|
|
ddd« | D ]s \ } }
|
|||
|
|
|
|||
|
|
| « }|j D �ci c]% }|j |j f|j" “Œ' }}|
|
|||
|
|
D ]' \ }}}||f|v r|||f |k( sJ ‚ Œe|
|
|||
|
|
sŒ t% |« ‚ Œu y# 1 sw Y Œ‚xY wc c}w )a$ Test that adding entities and resuming training works passably OK.
|
|||
|
|
There are two issues here:
|
|||
|
|
1) We have to re-add labels. This isn't very nice.
|
|||
|
|
2) There's no way to set the learning rate for the weight update, so we
|
|||
|
|
end up out-of-scale, causing it to learn too fast.
|
|||
|
|
ÚheyÚhowdyz hey thereÚhelloÚhiz i'm looking for a place to eatz,i'm looking for a place in the north of town)é é$ ÚLOCATIONzshow me chinese restaurants)rG é ÚCUISINEzshow me chines restaurants)rG é rw Úneré ÚentitiesN)r Úadd_pipeÚ add_labelÚ
|
|||
|
|
initializeÚrangeÚrandomÚshuffler
|
|||
|
|
Ú from_dictÚmake_docÚupdater Úto_diskr rS Ú
|
|||
|
|
start_charÚend_charÚlabel_Ú Exception)Ú
|
|||
|
|
TRAIN_DATArW ry Ú_ÚoffsetsÚstartÚendÚlabelÚitnÚraw_textÚentity_offsetsÚexampleÚ model_dirÚnlp2rX ÚentrS s rY Ú
|
|||
|
|
test_issue999r— Y s € ð
|
|||
|
|
�ˆ Ø �"ˆ
|
|||
|
|
Ø �bÐØ �"ˆ
|
|||
|
|
Ø
|
|||
|
|
ˆrˆ
|
|||
|
|
Ø )¨2Ð.Ø 7Ð:NÐ9OÐPØ &Ð);Ð(<Ð=Ø %Ð(:Ð';Ð<ð
|
|||
|
|
€Jô ‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€CÛ ‰
|
|||
|
|
ˆˆ7Û!(Ñ ˆE�3˜Ø �M‰M˜%Õ ñ ")ð !ð ‡N�NÔÜ�RŽyˆÜ�‰�zÔ"Û(2Ñ $ˆH�nÜ ×'Ñ'Ø— ‘ ˜XÓ&¨°^Ð(DóˆGð
|
|||
|
|
�J‰J˜�yÕ !ñ )3ð ô
|
|||
|
|
Œ ˜9Ø � ‰ �IÔ Ü# IÓ.ˆ÷
|
|||
|
|
ó %/Ñ ˆ�.Ù�8‹nˆØFIÇhÂhÓOÁh¸s�— ‘ § ¡ Ð.°·
|
|||
|
|
±
|
|||
|
|
Ñ:ÀhˆÐOÛ!/Ñ ˆE�3˜Ø�sˆ|˜tÑ#ؘU C˜LÑ)¨UÒ2Ð2Ð2Ùâ!Ü# D›/Ð)ñ
|
|||
|
|
"0ñ %/÷
|
|||
|
|
ˆ üò Ps Ã$ FÄ'*FÆF i2 c óŒ — ddddddœddddœdd ddœd
|
|||
|
|
d ddœd d
|
|||
|
|
ddœddddœddddœddddœddddœg g dœddddœdd ddœddddœd d ddœd d ddœd d!ddœd"d#ddœd$d
|
|||
|
|
ddœd%dddœd&dddœd'dddœd(d)ddœg g dœgd*d+d,œd-d.d,œgd/œd0dd1ddœdd2ddœdd3ddœd
|
|||
|
|
d4ddœd d5ddœdd6ddœdd7ddœdd8ddœdd9ddœddddœg
|
|||
|
|
g dœddddœgg dœgd*d.d,œd-d+d,œgd/œgd:œ} t « }g d;¢}t « 5 }|d<z }t | g« }t ||¬=« j « }|j d>« 5 }|j
|
|||
|
|
|« d d d « t |« }t ||« « } t | « dk( sJ ‚g }
|
|||
|
|
| D ]! } |
|
|||
|
|
j | j « « Œ# t |
|
|||
|
|
« d k( sJ ‚ d d d « y # 1 sw Y ŒwxY w# 1 sw Y y xY w)?Nr zRHow should I cook bacon in an oven?
|
|||
|
|
I've heard of people cooking bacon in an oven.ÚHowrB )ÚidÚorthry rD Úshouldr ÚIr4 Úcookr6 Úbaconr8 Úinr5 Úanr7 ÚovenrG ú?)ÚtokensÚbracketsé ú
|
|||
|
|
é
|
|||
|
|
é z'veé Úheardé
|
|||
|
|
Úofrx Úpeoplerv Úcookingé é é é rz r& ÚbakingrH )r� ÚvalueÚ
|
|||
|
|
not_bakingrI )ÚrawÚ sentencesrV z5What is the difference between white and brown eggs?
|
|||
|
|
ÚWhatr` ÚtheÚ
|
|||
|
|
differenceÚbetweenÚwhiteÚandÚbrownÚeggs©rš Ú
|
|||
|
|
paragraphs)ÚORTHÚ
|
|||
|
|
SENT_STARTÚENT_IOBÚENT_TYPEztest4402.spacy)ÚdocsÚattrsÚwb) r r r r Úto_bytesÚopenÚwriter ÚlistrT ÚextendÚ split_sents) Ú json_datarW rÈ ÚtmpdirÚ output_filerÇ ÚdataÚfile_ÚreaderÚ
|
|||
|
|
train_dataÚsplit_train_dataÚegs rY Útest_issue4402rÙ Š sà € ð ð mð $%¨e¸CÑ @Ø#$¨h¸sÑ CØ#$¨c¸#Ñ >Ø#$¨f¸SÑ AØ#$¨g¸cÑ BØ#$¨d¸3Ñ ?Ø#$¨d¸3Ñ ?Ø#$¨f¸SÑ AØ#$¨c¸#Ñ >ð
|
|||
|
|
#ð %'ñ
|
|||
|
|
ð $%¨d¸3Ñ ?Ø#%¨s¸3Ñ ?Ø#%¨u¸SÑ AØ#%¨w¸sÑ CØ#%¨t¸CÑ @Ø#%¨xÀÑ DØ#%¨yÀÑ EØ#%¨w¸sÑ CØ#%¨t¸CÑ @Ø#%¨t¸CÑ @Ø#%¨v¸cÑ BØ#%¨s¸3Ñ ?ð
|
|||
|
|
#ð %'ñ ð ðD '°Ñ5Ø *°SÑ9ðñG'
|
|||
|
|
ðR Pð $%¨f¸SÑ AØ#$¨d¸3Ñ ?Ø#$¨e¸CÑ @Ø#$¨lÀ3Ñ GØ#$¨iÀÑ DØ#$¨g¸cÑ BØ#$¨e¸CÑ @Ø#$¨g¸cÑ BØ#$¨f¸SÑ AØ#$¨c¸#Ñ >ð #ð %'ñð (*°4ÀÑ DÐ EÐSUÑVð! ð& '°Ñ5Ø *°SÑ9ðñ)
|
|||
|
|
ðSB
|
|||
|
|
ñE€IôL ‹)€CÚ 9€EÜ Œ ˜6Ø Ð /Ñ/ˆ ܘY˜KÓ(ˆÜ˜4 uÔ-×6Ñ6Ó8ˆØ
|
|||
|
|
×
|
|||
|
|
Ñ
|
|||
|
|
˜dÔ
|
|||
|
|
# uØ �K‰K˜Ô ÷ $ä˜ Ó$ˆÜ™& ›+Ó&ˆ
|
|||
|
|
Ü�:‹ !Ò#Ð#Ð#à ÐÛ ˆBØ × #Ñ # B§N¡NÓ$4Õ 5ð äÐ#Ó$¨Ò)Ð)Ñ)÷
|
|||
|
|
ˆ ÷ $Ð
|
|||
|
|
#ú÷
|
|||
|
|
ˆ ús% Ã(>F:Ä&F.Ä8A,F:Æ.F7 Æ3F:Æ:Ga
|
|||
|
|
[nlp]
|
|||
|
|
lang = "en"
|
|||
|
|
pipeline = ["tok2vec", "tagger"]
|
|||
|
|
|
|||
|
|
[components]
|
|||
|
|
|
|||
|
|
[components.tok2vec]
|
|||
|
|
factory = "tok2vec"
|
|||
|
|
|
|||
|
|
[components.tok2vec.model]
|
|||
|
|
@architectures = "spacy.Tok2Vec.v1"
|
|||
|
|
|
|||
|
|
[components.tok2vec.model.embed]
|
|||
|
|
@architectures = "spacy.MultiHashEmbed.v1"
|
|||
|
|
width = ${components.tok2vec.model.encode:width}
|
|||
|
|
attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
|
|||
|
|
rows = [5000,2500,2500,2500]
|
|||
|
|
include_static_vectors = false
|
|||
|
|
|
|||
|
|
[components.tok2vec.model.encode]
|
|||
|
|
@architectures = "spacy.MaxoutWindowEncoder.v1"
|
|||
|
|
width = 96
|
|||
|
|
depth = 4
|
|||
|
|
window_size = 1
|
|||
|
|
maxout_pieces = 3
|
|||
|
|
|
|||
|
|
[components.tagger]
|
|||
|
|
factory = "tagger"
|
|||
|
|
|
|||
|
|
[components.tagger.model]
|
|||
|
|
@architectures = "spacy.Tagger.v2"
|
|||
|
|
nO = null
|
|||
|
|
|
|||
|
|
[components.tagger.model.tok2vec]
|
|||
|
|
@architectures = "spacy.Tok2VecListener.v1"
|
|||
|
|
width = ${components.tok2vec.model.encode:width}
|
|||
|
|
upstream = "*"
|
|||
|
|
iu c óx ‡
|
|||
|
|
— ddg d¢ifddg d¢ifg} t j t t « « }g Š
|
|||
|
|
| D ]< }‰
|
|||
|
|
j t j |j |d « |d « « Œ> |j ˆ
|
|||
|
|
fd„¬ « }t d
|
|||
|
|
« D ] }i }|j ‰
|
|||
|
|
||¬ « Œ g d ¢}t |j |d¬
|
|||
|
|
« « }t |j |d¬
|
|||
|
|
« « }|dd D � cg c] } | d j ‘Œ c} |dd D � cg c] } | d j ‘Œ c} k( sJ ‚yc c} w c c} w )z<Test that an empty document doesn't mess up an entire batch.zI like green eggsrM )ÚNÚVÚJrÛ z Eat blue ham)rÜ rÝ rÛ r rD c ó •— ‰ S rm rj )Útrain_exampless €rY ú<lambda>z test_issue7029.<locals>.<lambda> s ø€ ±NrZ )Ú get_examplesé2 ©ÚsgdÚlosses)ÚfirstÚsecondÚthirdÚfourthr¾ ÚthenÚsomer3 )Ú
|
|||
|
|
batch_sizer6 Néÿÿÿÿ)r Ú from_configr Ú CONFIG_7029Úappendr
|
|||
|
|
r‚ rƒ r~ r r„ rÍ ÚpipeÚtag_) rŠ rW ÚtÚ optimizerÚirå ÚtextsÚdocs1Údocs2rX rß s @rY Útest_issue7029rù
|
|||
|
|
s6 ø€ ð
|
|||
|
|
˜vÒ';Ð <Ð=Ø ˜&¢/Ð2Ð3ð€Jô ×
|
|||
|
|
Ñ
|
|||
|
|
Ô 2´;Ó ?Ó
|
|||
|
|
@€CØ€NÛ
|
|||
|
|
ˆØ× Ñ œg× /Ñ /°· ± ¸Q¸q¹TÓ0BÀAÀaÁDÓ IÕJð à—‘Ó,B�ÓC€IÜ
|
|||
|
|
�2ŽYˆØˆØ �
|
|||
|
|
‰
|
|||
|
|
�> y¸ˆ
|
|||
|
|
Õ@ð ò
|
|||
|
|
N€EÜ �—‘˜%¨A�Ó.Ó /€EÜ �—‘˜%¨A�Ó.Ó /€EØ#(¨¨"¡:Ó .¡:˜CˆC�‰F�K‹K :Ñ .È%ÐPSÐQSÉ*Ó2UÉ*À3°3°q±6·;³;È*Ñ2UÒ UÐUÑ UùÒ .ùÒ2Us Ã5D2ÄD7c óŠ — g d¢}g d¢}t | ||¬« }t d« t d« dfg}t ||« }|g d¢k( sJ ‚y )N)r� r r! r% r& )TTTFT©rL rh ú
|
|||
|
|
I flew to zI flew to LondonÚLOC©rB rB rB úU-LOCrB ©r rT r ©Úen_vocabrL rh rX r{ rM s rY Útest_gold_biluo_Ur " sO € Ú .€EÚ
|
|||
|
|
,€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CÜ�\Ó"¤CÐ(:Ó$;¸UÐCÐD€HÜ hÓ /€DØ Ò0Ò 0Ð0Ñ 0rZ c óŠ — g d¢}g d¢}t | ||¬« }t d« t d« dfg}t ||« }|g d¢k( sJ ‚y )N)r� r r! ÚSanÚ Franciscor& )TTTTFTrû rü úI flew to San Franciscorý )rB rB rB rE úL-LOCrB r r s rY Útest_gold_biluo_BLr + sO € Ú 8€EÚ
|
|||
|
|
2€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CÜ�\Ó"¤CÐ(AÓ$BÀEÐJÐK€HÜ hÓ /€DØ Ò9Ò 9Ð9Ñ 9rZ c óŠ — g d¢}g d¢}t | ||¬« }t d« t d« dfg}t ||« }|g d¢k( sJ ‚y )N©r� r r! r r r# r& ©TTTTTFTrû rü ú I flew to San Francisco Valleyrý )rB rB rB rE rF r rB r r s rY Útest_gold_biluo_BILr 4 sO € Ú B€EÚ
|
|||
|
|
8€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CÜ�\Ó"¤CÐ(HÓ$IÈ5ÐQÐR€HÜ hÓ /€DØ ÒBÒ BÐBÑ BrZ c ó — g d¢}g d¢}t | ||¬« }t d« t d« dft d« t d« dfg}t j t « 5 t ||« d d d « y # 1 sw Y y xY w)Nr r rû rü r
|
|||
|
|
rý r )r rT ÚpytestÚraisesÚ
|
|||
|
|
ValueErrorr )r rL rh rX r{ s rY Útest_gold_biluo_overlapr = so € Ú B€EÚ
|
|||
|
|
8€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€Cä ˆ\Ó œCÐ @Ó AÀ5ÐIÜ ˆ\Ó œCÐ 9Ó :¸EÐBð€Hô
|
|||
|
|
� ‰ ”zÕ "Ü ˜c 8Ô,÷
|
|||
|
|
#× "Ñ "ús Á
|
|||
|
|
A4Á4A=c óæ — g d¢}g d¢}t | ||¬« }t d« t d« dfg}t j t « 5 t ||« }d d d « g d¢k( sJ ‚y # 1 sw Y ŒxY w)N)r� r r! r r zValley.)TTTTTFrû rü r
|
|||
|
|
rý )rB rB rB ú-r r )r rT r ÚwarnsÚ UserWarningr r s rY Útest_gold_biluo_misalignr I si € Ú >€EÚ
|
|||
|
|
2€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CÜ�\Ó"¤CÐ(HÓ$IÈ5ÐQÐR€HÜ � ‰ ”kÕ "Ü$ S¨(Ó3ˆ÷
|
|||
|
|
#à Ò1Ò 1Ð1Ñ 1÷
|
|||
|
|
#Ð "ús Á
|
|||
|
|
A'Á'A0c ó@ — g d¢}g d¢}|D �cg c] }| j j |« ‘Œ }}t | |¬« }t | |¬« }|j dt j
|
|||
|
|
|d¬« « }t
|
|||
|
|
||« }|j dd¬« }|g d¢k( sJ ‚y c c}w ) N©r� ÚlikeÚstuff©r. r/ r. ©rL ÚTAGÚuint64)ÚdtypeT©Ú as_string)ÚstringsÚaddr Ú
|
|||
|
|
from_arrayÚnumpyÚarrayr
|
|||
|
|
Ú get_aligned)r rL rM ÚtagÚtag_idsÚ predictedÚ referencer“ s rY Útest_example_constructorr. S s› € Ú "€EÚ #€DÙ48Ó9±D¨Sˆx× Ñ ×#Ñ# CÕ(°D€GÐ9Ü�H EÔ*€IÜ�H EÔ*€IØ×$Ñ$ U¬E¯K©K¸ÀxÔ,PÓQ€IÜ�i Ó+€GØ × Ñ ˜u°Ð Ó 5€DØ Ò+Ò +Ð+Ñ +ùò
|
|||
|
|
:s �"Bc ó– — g d¢}g d¢}t | |¬« }t j |d|i« }|j dd¬« }|g d¢k( sJ ‚y )Nr r r ÚTAGSr Tr" )r r
|
|||
|
|
r‚ r) )r rL rM r, r“ s rY Útest_example_from_dict_tagsr1 _ sR € Ú "€EÚ #€DÜ�H EÔ*€IÜ× Ñ ¨F°D¨>Ó:€GØ × Ñ ˜u°Ð Ó 5€DØ Ò+Ò +Ð+Ñ +rZ c ó’ — g d¢}g d¢}t | ||¬« }t j |d|i« }|j « }|g d¢k( sJ ‚y )N©ÚaÚbÚcÚd©TTFTrû rL )NNNN©r r
|
|||
|
|
r‚ Úget_aligned_ner©r rL rh r, r“ Úner_tagss rY Ú test_example_from_dict_no_nerr= h sN € Ú €EÚ
|
|||
|
|
&€FÜ�H E°&Ô9€IÜ× Ñ ¨G°UÐ+;Ó<€GØ×&Ñ&Ó(€HØ Ò/Ò /Ð/Ñ /rZ c ó˜ — g d¢}g d¢}t | ||¬« }t j ||g d¢dœ« }|j « }|g d¢k( sJ ‚y )Nr3 r8 rû )rÿ NNN©rL r{ r9 r; s rY Ú test_example_from_dict_some_nerr@ q sT € Ú €EÚ
|
|||
|
|
&€FÜ�H E°&Ô9€IÜ× Ñ Ø˜UÒ0KÑLó€Gð ×&Ñ&Ó(€HØ Ò2Ò 2Ð2Ñ 2rZ zignore::UserWarningc ó — ddddddddœdddd dœd
|
|||
|
|
d d d
|
|||
|
|
dœdddddœdddddœgigigdœg}t t |« « }t |« dk( sJ ‚|D ] }|j d« sŒJ ‚ D ] }|j d k( rŒJ ‚ t t
|
|||
|
|
|j |D �cg c] }|j ‘Œ c}|D �cg c] }t |j « ‘Œ c}¬« |« }|j « }|g d¢k( sJ ‚y c c}w c c}w )NrD r¸ r¤ Únnr' zMs.)ÚdepÚheadr* r› r; ÚHaagr< r ÚVBZÚplaysÚdobjrí ÚEliantir@ éþÿÿÿr& rÁ rÅ rû )NNNNN) rÍ r rT Úhas_annotationÚent_iobr
|
|||
|
|
r rU ÚtextÚboolÚ whitespace_r: )r rÓ rÇ rX ÚtokenÚwrØ r< s rY Útest_json_to_docs_no_nerrR | sX € ð ð à $Ø(,°aÀÈuÑ Uà+2Ø,-Ø+0Ø,2ñ !"ð ,2Ø,-Ø+0Ø,3ñ !"ð ,2Ø,.Ø+0Ø,5ñ !"ð )0¸ÀCÐQTÑ Uð)' ðð"ðð ñ
|
|||
|
|
ð" €DôF ” ˜TÓ"Ó #€DÜ ˆt‹9˜Š>Ј>ÛˆØ×%Ñ% iÕ0Ð0Ð0ð ãˆØ�}‰} Ó!Ð!Ð!ð ä Ü Ø �I‰IÙ#&Ó'¡3˜a�1—6“6 3Ñ'Ù14Ó5±¨A”D˜Ÿ ™ Õ'°Ñ5ô
|
|||
|
|
ð
|
|||
|
|
ó
|
|||
|
|
|
|||
|
|
€Bð ×!Ñ!Ó#€HØ Ò5Ò 5Ð5Ñ 5ùò
|
|||
|
|
(ùÚ5s ÂC6Â3 C;c ó0 — g d¢}g d¢}g d¢}t | |¬« }t j |||dœ« }|j dk( sJ ‚|j « }t |« dk( sJ ‚|d j d k( sJ ‚|d
|
|||
|
|
j d k( sJ ‚g d ¢}g d
|
|||
|
|
¢}g d¢}t | |¬« }t j |||dœ« }|j dk( sJ ‚|j « }t |« dk( sJ ‚|d j d k( sJ ‚|d
|
|||
|
|
j d k( sJ ‚y )N)r� r r! úSan Francisco ValleyÚhadz loads of fun)
|
|||
|
|
r� r r! r r r# rU Úloadsr Úfun)
|
|||
|
|
TFFFFFTFFFr )rL ri z0I flew to San Francisco Valley had loads of fun r r z I flew to San Francisco Valley rD zhad loads of fun ) r� r r! r r r# rU rV zof fun)r� r r! ú
|
|||
|
|
San Franciscor# rU zloads ofrW )TFFFFTFF)r r
|
|||
|
|
r‚ rM rÏ rT )r rL Ú
|
|||
|
|
gold_wordsri rX r“ Úsplit_exampless rY Útest_split_sentencesr[ ³ s6 € â N€EÚ_€JÚV€Kä
|
|||
|
|
|
|||
|
|
ˆh˜eÔ
|
|||
|
|
$€CÜ× Ñ ¨zÈ+Ñ%VÓW€GØ �<‰<ÐMÒ MÐMÐ MØ ×(Ñ(Ó*€NÜ ˆ~Ó !Ò #Ð#Ð #Ø ˜!Ñ × !Ñ !Ð%FÒ FÐFÐ FØ ˜!Ñ × !Ñ !Ð%8Ò 8Ð8Ð 8â W€EÚY€JÚH€Kä
|
|||
|
|
|
|||
|
|
ˆh˜eÔ
|
|||
|
|
$€CÜ× Ñ ¨zÈ+Ñ%VÓW€GØ �<‰<ÐMÒ MÐMÐ MØ ×(Ñ(Ó*€NÜ ˆ~Ó !Ò #Ð#Ð #Ø ˜!Ñ × !Ñ !Ð%FÒ FÐFÐ FØ ˜!Ñ × !Ñ !Ð%8Ò 8Ð8Ñ 8rZ c ór — g d¢}g d¢}t | ||¬« }d}t |« t |dz « dfg}g d¢}t j |||dœ« }|j « } | g d ¢k( sJ ‚t d
|
|||
|
|
« t d « d ft |« t |dz « dfg}g d
|
|||
|
|
¢}t j |||dœ« }|j « } | g d¢k( sJ ‚t d
|
|||
|
|
« t d« d ft |« t |dz « dfg}g d
|
|||
|
|
¢}t j |||dœ« }|j « } | g d¢k( sJ ‚y )N)úMr and ú Mrs Smithúflew torT r& ©TTTFFrû úMr and Mrs Smith flew to rT rý )úMr and Mrs Smithr r! r r r# r& r? rþ r] rb ÚPERSON© úMr andÚMrsÚSmithr r! r r r# r& )rB úU-PERSONrB rÿ rB ú
|
|||
|
|
Mr and Mrs)rB NrB rÿ rB ©r rT r
|
|||
|
|
r‚ r: ©
|
|||
|
|
r Ú en_tokenizerrL rh rX Úprefixr{ rY r“ r< s
|
|||
|
|
rY Útest_gold_biluo_one_to_manyrn Î sY € Ú L€EÚ
|
|||
|
|
-€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CØ
|
|||
|
|
(€FÜ�V“œc &Ð+AÑ"AÓ BÀEÐJÐK€HÚV€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ Ò4Ò 4Ð4Ð 4ô
|
|||
|
|
|
|||
|
|
ˆY‹ œÐ /Ó0°(Ð;Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò
|
|||
|
|
]€Jä× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ Ò;Ò ;Ð;Ð ;ô
|
|||
|
|
|
|||
|
|
ˆY‹ œ˜\Ó*¨HÐ5Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò
|
|||
|
|
]€Jä× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ Ò5Ò 5Ð5Ñ 5rZ c ó¨ — g d¢}g d¢}t | ||¬« }d}t |« t |dz « dfg}g d¢}t j |||dœ« }|j « } | g d ¢k( sJ ‚t d
|
|||
|
|
« t d « d ft |« t |dz « dfg}g d
|
|||
|
|
¢}t j |||dœ« }|j « } g d¢}
|
|||
|
|
| |
|
|||
|
|
k( sJ ‚y )Nrd ) TTTTTTTFFrû ra rT rý )rb r_ rT r& r? ) rB rB rB rB rB rE rF r rB r] rb rc )re r^ r_ rT r& ) rB rC zL-PERSONrB rB rE rF r rB rj ) r rl rL rh rX rm r{ rY r“ r< Úexpecteds rY Útest_gold_biluo_many_to_onerq ð sì € Ú W€EÚ
|
|||
|
|
E€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CØ
|
|||
|
|
(€FÜ�V“œc &Ð+AÑ"AÓ BÀEÐJÐK€HÚM€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ ÒPÒ PÐPÐ Pô
|
|||
|
|
|
|||
|
|
ˆY‹ œÐ /Ó0°(Ð;Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò Q€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HÚV€HØ �xÒ Ð Ñ rZ c ó¤ — g d¢}g d¢}t | ||¬« }d}t |« t |dz « dfg}g d¢}t j |||dœ« }|j « } | g d ¢k( sJ ‚t d
|
|||
|
|
« t d « d ft |« t |dz « dfg}g d
|
|||
|
|
¢}t j |||dœ« }|j « } | g d¢k( sJ ‚y )N)ri rg r r! rX r# r& )TTTTTFFrû ra rT rý )ÚMrz
|
|||
|
|
and Mrs Smithr_ r úFrancisco Valleyr& r? ©rB rB rB rB rE r rB r] rb rc )re r^ r_ r rt r& )NNrB rB rE r rB rj rk s
|
|||
|
|
rY Útest_gold_biluo_misalignedrv sè € Ú Q€EÚ
|
|||
|
|
9€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CØ
|
|||
|
|
(€FÜ�V“œc &Ð+AÑ"AÓ BÀEÐJÐK€HÚS€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ ÒBÒ BÐBÐ Bô
|
|||
|
|
|
|||
|
|
ˆY‹ œÐ /Ó0°(Ð;Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò T€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ×&Ñ&Ó(€HØ ÒDÒ DÐDÑ DrZ c óò — t g d¢d« \ }}t | ||¬« }d}t |« t |dz « dfg}g d¢}g d¢}t j ||||d œ« } | j « }
|
|||
|
|
|
|
|||
|
|
g d
|
|||
|
|
¢k( sJ ‚y ) N)r� r r! rX r# r& z I flew to San Francisco Valley.rû z I flew to rT rý )r� r ú r! rT r& )TTFTFF©rL rh r{ ru )r r rT r
|
|||
|
|
r‚ r: ) r rl rL rh rX rm r{ rY Ú gold_spacesr“ r< s rY Ú%test_gold_biluo_additional_whitespacer{ sŽ € ä(Ú;Ø*ó�M€Eˆ6ô ˆh˜e¨FÔ
|
|||
|
|
3€CØ
|
|||
|
|
€FÜ�V“œc &Ð+AÑ"AÓ BÀEÐJÐK€HÚF€JÚ9€KÜ× Ñ Ø �z¨[ÀhÑ
|
|||
|
|
Oó€Gð ×&Ñ&Ó(€HØ ÒBÒ BÐBÑ BrZ c ó — |d« }g d¢}g d¢}dg}t j ||||dœ« }|j « }|g d¢k( sJ ‚ |d« }g d¢}g d¢}dg}t j ||||dœ« }|j « }|g d ¢k( sJ ‚y )
|
|||
|
|
NzI'll return the A54 amount)r� ú'llÚreturnrº ÚAÚ54Úamount)FTTTFTF)r° r³ ÚMONEYry )rB rB rB rB zU-MONEYrB zI'll return the $54 amount)r� r} r~ rº ú$r€ r� )rB rB rB rB zB-MONEYzL-MONEYrB )r
|
|||
|
|
r‚ r: )r rl rX rY rz r{ r“ r< s rY Útest_gold_biluo_4791r„ - s² € Ù
|
|||
|
|
Ð3Ó
|
|||
|
|
4€CÚC€JÚ?€KØ!Ð"€HÜ× Ñ Ø �z¨[ÀhÑ
|
|||
|
|
Oó€Gð ×&Ñ&Ó(€HØ Ò;Ò ;Ð;Ð ;á
|
|||
|
|
Ð3Ó
|
|||
|
|
4€CÚC€JÚ?€KØ!Ð"€HÜ× Ñ Ø �z¨[ÀhÑ
|
|||
|
|
Oó€Gð ×&Ñ&Ó(€HØ ÒFÒ FÐFÑ FrZ c ó¨ — d}g d¢}ddg} | |« }t ||« }||k( sJ ‚t ||« }|D �cg c]
|
|||
|
|
}|d sŒ |‘Œ }}||k( sJ ‚y c c}w )Nú$I flew to Silicon Valley via London.©rB rB rB rE r rB úU-GPErB )r¨ é rý )é é# ÚGPEr )r r )rl rM Ú
|
|||
|
|
biluo_tagsrŒ rX Úbiluo_tags_convertedÚoffsets_convertedr– s rY Ú'test_roundtrip_offsets_biluo_conversionr� C sv € Ø 1€DÚE€JØ Ð0€GÙ
|
|||
|
|
�tÓ
|
|||
|
|
€CÜ0°°gÓ>ÐØ :Ò -Ð-Ð -Ü-¨c°:Ó>ÐÙ(/Ó:© °3°q³6š¨ÐÐ:Ø Ò 'Ð'Ñ 'ùò ;s ¶
|
|||
|
|
AÁAc ó6 — | d« }g d¢}t ||« }|D �cg c] }|j sŒ|‘Œ }}t |« dk( sJ ‚|d j dk( sJ ‚|d j dk( sJ ‚|d j dk( sJ ‚|d j d k( sJ ‚y c c}w )
|
|||
|
|
Nr† r‡ r r zSilicon Valleyrý rD r% rŒ )r rˆ rT rM )rl rX r� ÚspansÚspans rY Útest_biluo_spansr” O s¦ € Ù
|
|||
|
|
Ð=Ó
|
|||
|
|
>€CÚE€JÜ ZÓ 0€EÙ #Ó 3™e�d t§{£{ŠT˜e€EÐ 3Ü ˆu‹:˜Š?Ј?Ø �‰8�=‰=Ð ,Ò ,Ð,Ð ,Ø �‰8�?‰?˜eÒ #Ð#Ð #Ø �‰8�=‰=˜HÒ $Ð$Ð $Ø �‰8�?‰?˜eÒ #Ð#Ñ #ùò
|
|||
|
|
4s
|
|||
|
|
�B¯Bc óÔ — g d¢}g d¢}t | ||¬« }d}dt d« dft |« t |dz « d fg}g d
|
|||
|
|
¢}t j |||d œ« }|j j
|
|||
|
|
} | D �
|
|||
|
|
cg c] }
|
|||
|
|
|
|
|||
|
|
j |
|
|||
|
|
j f‘Œ c}
|
|||
|
|
d d
|
|||
|
|
gk( sJ ‚|j | « } | D �
|
|||
|
|
cg c] }
|
|||
|
|
|
|
|||
|
|
j |
|
|||
|
|
j f‘Œ c}
|
|||
|
|
ddgk( sJ ‚y c c}
|
|||
|
|
w c c}
|
|||
|
|
w )N)rb r r! rT r& r` rû ra r rb rc rT rý )
|
|||
|
|
rs r¾ rf rg r r! r r r# r& r? ©r r6 ©r5 r¦ )r rD )r4 r6 ) r rT r
|
|||
|
|
r‚ r- rS r� rŽ Úget_aligned_spans_y2x) r rl rL rh rX rm r{ Ú
|
|||
|
|
tokens_refr“ Úents_refr– Úents_y2xs rY Útest_aligned_spans_y2xrœ [ sõ € Ú K€EÚ
|
|||
|
|
-€FÜ
|
|||
|
|
|
|||
|
|
ˆh˜e¨FÔ
|
|||
|
|
3€CØ
|
|||
|
|
(€Fà
|
|||
|
|
ŒCÐ"Ó # XÐ.Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò
|
|||
|
|
`€Jä× Ñ ¨zÀxÑ%PÓQ€GØ× Ñ ×%Ñ%€HÙ,4Ó 5©H SˆS�Y‰Y˜Ÿ™Ò ¨HÑ 5¸&À&Ð9IÒ IÐIÐ IØ×,Ñ,¨XÓ6€HÙ,4Ó 5©H SˆS�Y‰Y˜Ÿ™Ò ¨HÑ 5¸&À&Ð9IÒ IÐIÑ Iùò 6ùâ 5s Á8 C Â6 C%c ó, — d}t « }dddœdddœg}|j d« }|j |« ||« }|j D �cg c] }|j |j
|
|||
|
|
f‘Œ c}dd gk( sJ ‚d
|
|||
|
|
}d t
|
|||
|
|
d« dft
|
|||
|
|
|« t
|
|||
|
|
|dz « dfg} g d ¢}
|
|||
|
|
t j ||
|
|||
|
|
| d
|
|||
|
|
œ« } | j j D �cg c] }|j |j
|
|||
|
|
f‘Œ c}ddgk( sJ ‚| j j } | D �cg c] }|j |j
|
|||
|
|
f‘Œ c}dd gk( sJ ‚| j | « }
|
|||
|
|
|
|
|||
|
|
D �cg c] }|j |j
|
|||
|
|
f‘Œ c}ddgk( sJ ‚y c c}w c c}w c c}w c c}w )Nz-Mr and Mrs Smith flew to San Francisco Valleyrc rb )r� Úpatternrý rT Ú entity_rulerr– r— ra r )ri rg r r! rX r# r? )r r )r6 r5 ) r r| Ú add_patternsrS r� rŽ rT r
|
|||
|
|
r‚ r- r, Úget_aligned_spans_x2y)r rl rM rW ÚpatternsÚrulerrX r– rm r{ r™ r“ Ú ents_predÚents_x2ys rY Útest_aligned_spans_x2yr¦ n s¥ € Ø :€DÜ
|
|||
|
|
‹)€CàÐ'9Ñ:ØÐ$:Ñ;ð€Hð
|
|||
|
|
�L‰L˜ Ó (€EØ ×Ñ�xÔ Ù
|
|||
|
|
|
|||
|
|
ˆd‹)€CØ,/¯HªHÓ 5©H SˆS�Y‰Y˜Ÿ™Ò ¨HÑ 5¸&À&Ð9IÒ IÐIÐ IØ
|
|||
|
|
(€Fà
|
|||
|
|
ŒCÐ"Ó # XÐ.Ü ˆV‹”c˜&Ð#9Ñ9Ó:¸EÐBð€Hò R€JÜ× Ñ ¨zÀxÑ%PÓQ€GØ,3×,=Ñ,=×,BÒ,BÓ CÑ,B SˆS�Y‰Y˜Ÿ™Ò Ð,BÑ CÈÐPVÐGWÒ WÐWÐ Wà×!Ñ!×&Ñ&€IÙ,5Ó 6©I SˆS�Y‰Y˜Ÿ™Ò ¨IÑ 6¸6À6Ð:JÒ JÐJÐ JØ×,Ñ,¨YÓ7€HÙ,4Ó 5©H SˆS�Y‰Y˜Ÿ™Ò ¨HÑ 5¸&À&Ð9IÒ IÐIÑ Iùò 6ùò Dùò 7ùâ 5s Á Fà FÄ F Å Fc ó — d}t « } ||« }|j |« }g }d}|j |j t |« t |dz « d¬« « |j |j t |« t |dz « d¬« « d}||j
|
|||
|
|
|< t
|
|||
|
|
||« } | j j
|
|||
|
|
| }
|
|||
|
|
|
|
|||
|
|
D � cg c] } | j | j f‘Œ c} d d
|
|||
|
|
gk( sJ ‚| j |
|
|||
|
|
d ¬ « } | D � cg c] } | j | j f‘Œ c} d gk( sJ ‚| j |
|
|||
|
|
d
|
|||
|
|
¬ « }
|
|||
|
|
|
|
|||
|
|
D � cg c] } | j | j f‘Œ c} d d
|
|||
|
|
gk( sJ ‚y c c} w c c} w c c} w )Nr
|
|||
|
|
rü rX ÚCITY)r� rT ÚVALLEYÚ overlap_ents)r4 r8 )r4 r5 F)Ú
|
|||
|
|
allow_overlapT) r rƒ rð Ú char_spanrT r’ r
|
|||
|
|
r- r� rŽ r˜ )r rl rM rW rX Úgold_docr’ rm Ú spans_keyr“ Ú
|
|||
|
|
spans_goldr– Úspans_y2x_no_overlapÚspans_y2x_overlaps rY Ú test_aligned_spans_y2x_overlapr² ˆ sŸ € Ø +€DÜ
|
|||
|
|
‹)€CÙ
|
|||
|
|
|
|||
|
|
ˆd‹)€Cà�|‰|˜DÓ!€HØ €EØ
|
|||
|
|
€FØ ‡L�LØ×Ñœ3˜v›;¬¨F°_Ñ,DÓ(EÈVÐÓTôð
|
|||
|
|
‡L�LØ×ÑÜ �‹Kœ˜VÐ&<Ñ <Ó=ÀXð ó
|
|||
|
|
ôð
|
|||
|
|
€IØ %€H‡N�N�9Ñ Ü�c˜8Ó$€GØ×"Ñ"×(Ñ(¨Ñ3€JÙ,6Ó 7©J SˆS�Y‰Y˜Ÿ™Ò ¨JÑ 7¸FÀFÐ;KÒ KÐKÐ Kð #×8Ñ8Ø %ð 9ó Ðñ -AÓ AÑ,@ SˆS�Y‰Y˜Ÿ™Ò Ð,@Ñ AÀfÀXÒ MÐMÐ MØ ×5Ñ5°jÐPTÐ5ÓUÐÙ,=Ó >Ñ,= SˆS�Y‰Y˜Ÿ™Ò Ð,=Ñ >À6È6ÐBRÒ RÐRÑ Rùò 8ùò Bùâ >s à E=Ä FÅ Fc ó| — | d« }g d¢}t j |d|i« }|j d« g d¢k( sJ ‚y )Nr† )NrB rB rE r rB rˆ rB r{ rÅ )r r r r4 rD r r4 r )r
|
|||
|
|
r‚ r) )rl rX r� r“ s rY Útest_gold_ner_missing_tagsr´ § sC € Ù
|
|||
|
|
Ð=Ó
|
|||
|
|
>€CÚF€JÜ× Ñ j°*Ð%=Ó>€GØ × Ñ ˜yÓ )Ò-EÒ EÐEÑ ErZ c óZ — | d« }g d¢}dgt |« z }t j |||dœ« }|j d¬« \ }}|j d¬« \ }}|g d¢k( sJ ‚|g d¢k( sJ ‚ | d « }d
|
|||
|
|
g}dg}t j |||dœ« }|j d¬« \ }}||k( sJ ‚||k( sJ ‚t |j
|
|||
|
|
d gdgd gd
|
|||
|
|
g¬
|
|||
|
|
« } t |j
|
|||
|
|
g d¢g d¢g d¢g d¢¬
|
|||
|
|
« }
|
|||
|
|
t | |
|
|||
|
|
« }|j d¬« \ }} |d gk( sJ ‚| d gk( sJ ‚y )Nz He pretty quickly walks away)r4 r r4 r4 r rC )rP rQ T)Ú projectivizeF)r4 r r4 r4 r4 ÚConrailr zDouble-Jointedr< )rL rh rQ rP )ÚDoubler ÚJointed)TTT)Úamodr@ r< )r r r )rT r
|
|||
|
|
r‚ Úget_aligned_parser rU ) rl rX rP rQ r“ Ú
|
|||
|
|
proj_headsÚ proj_labelsÚ
|
|||
|
|
nonproj_headsÚnonproj_labelsÚdoc_aÚdoc_bÚ proj_depss rY Útest_projectivizerà ® sv € Ù
|
|||
|
|
Ð5Ó
|
|||
|
|
6€CÚ €EØ ˆ7”S˜“ZÑ €DÜ× Ñ ¨u¸dÑ%CÓD€GØ %× 7Ñ 7ÀTÐ 7Ó JÑ€J� Ø$+×$=Ñ$=È5Ð$=Ó$QÑ!€M�>Ø š Ò (Ð(Ð (Ø šOÒ +Ð+Ð +ñ �yÓ
|
|||
|
|
!€CØ
|
|||
|
|
ˆC€EØ ˆ7€DÜ× Ñ ¨u¸dÑ%CÓD€GØ %× 7Ñ 7ÀTÐ 7Ó JÑ€J� Ø ˜Ò Ð Ð Ø ˜$Ò Ð Ð ô
|
|||
|
|
Ø � ‰ Ð*Ð+°U°GÀ6À(ÐSTÐRUô
|
|||
|
|
€Eô
|
|||
|
|
Ø � ‰ Ú(Ú!Ú
|
|||
|
|
&Úô
|
|||
|
|
€Eô �e˜UÓ#€GØ #× 5Ñ 5À4Ð 5Ó HÑ€J� Ø ˜$˜Ò Ð Ð Ø ˜˜Ò Ð Ñ rZ c ó² — g d¢} g d¢}g d¢}t | « }||k( sJ ‚t j t « 5 t |« d d d « y # 1 sw Y y xY w)N)rB rB rE rF rB rC )rB rB rE r rB rh )rB rB ú"rE rF )r r r r )Úgood_iobÚ
|
|||
|
|
good_biluoÚbad_iobÚconverted_biluos rY Útest_iob_to_biluorÊ Ò sI € Ú<€HÚ>€JÚ/€GÜ" 8Ó,€OØ ˜ Ò (Ð(Ð (Ü � ‰ ”zÕ "Ü�WÔ ÷
|
|||
|
|
#× "Ñ "ús ¸ A
|
|||
|
|
Á
|
|||
|
|
Ac óH — | j }| D �cg c] }|j ‘Œ }}| D �cg c] }|j ‘Œ }}| D �cg c] }|j ‘Œ }}| D �cg c] }t |j
|
|||
|
|
« ‘Œ }}| D �cg c] }|j ‘Œ }}| D �cg c] }|j ‘Œ }}| D �cg c] }|j j ‘Œ } }| j }
|
|||
|
|
| j D � cg c]% } | j | j | j f‘Œ' } } t « 5 }
|
|||
|
|
t! « }|
|
|||
|
|
dz }t# j$ |t' | « g« |
|
|||
|
|
dz }t) | g¬« j+ |« t- |« }t/ ||« « }d d d « t1 | « t3 d„ D « « k( sJ ‚|d }||j4 j k( sJ ‚||j4 D �cg c] }|j ‘Œ c}k( sJ ‚||j4 D �cg c] }|j ‘Œ c}k( sJ ‚||j4 D �cg c] }|j ‘Œ c}k( sJ ‚||j4 D �cg c] }t |j
|
|||
|
|
« ‘Œ c}k( sJ ‚||j4 D �cg c] }|j ‘Œ c}k( sJ ‚||j4 D �cg c] }|j ‘Œ c}k( sJ ‚| |j4 D �cg c] }|j j ‘Œ c}k( sJ ‚| |j4 j D � cg c]% } | j | j | j f‘Œ' c} k( sJ ‚d|j4 j v sJ ‚d|j4 j v sJ ‚|
|
|||
|
|
d |j4 j d k( sJ ‚|
|
|||
|
|
d |j4 j d k( sJ ‚y c c}w c c}w c c}w c c}w c c}w c c}w c c}w c c} w # 1 sw Y �ŒexY wc c}w c c}w c c}w c c}w c c}w c c}w c c}w c c} w )Nzroundtrip.jsonzroundtrip.spacy)rÇ c 3 ó2 K — | ] }t |« –— Œ y wrm )rT )Ú.0rØ s rY ú <genexpr>z0test_roundtrip_docs_to_docbin.<locals>.<genexpr>ñ s è ø€ Ð?Ñ-> rœ3˜rŸ7Ñ->ùs ‚r rJ rK )rM Úidxrò Úpos_ÚstrÚmorphÚlemma_Údep_rD rõ rV rS r† r‡ rˆ r r ÚsrslyÚ
|
|||
|
|
write_jsonr
|
|||
|
|
r r… r rÍ rT Úsumr- )rX rM ró rÏ rM rN rO rR rQ rP rV ÚerS rÑ Ú reloaded_nlpÚ json_filerÒ rÕ Úreloaded_examplesÚreloaded_examples rY Ú test_roundtrip_docs_to_docbinrÝ Ü sØ € Ø �8‰8€DÙ Ó
|
|||
|
|
™#�Qˆ1�5‹5˜#€CÐ
|
|||
|
|
Ù Ó ™C�qˆA�F‹F˜C€DÐ Ù Ó
|
|||
|
|
™3�aˆ1�6‹6˜3€CÐ
|
|||
|
|
Ù$'Ó
|
|||
|
|
(¡C˜qŒc�!—'‘'�l C€FÐ
|
|||
|
|
(Ù #Ó
|
|||
|
|
$¡˜1ˆa�h‹h €FÐ
|