241 lines
41 KiB
Plaintext
241 lines
41 KiB
Plaintext
|
Ë
|
|||
|
|
?û g4u ã ód — d dl Z d dlZd dlZd dlmZ d dlmZmZ d dlm Z d dl
|
|||
|
|
m Z d dl m
|
|||
|
|
Z
|
|||
|
|
d dlmZ d dlmZ d d lmZ d d
|
|||
|
|
lmZ d d lmZ d d lmZmZ d d
|
|||
|
|
lm Z m Z m Z d dl m Z ddlm!Z! dddgifddddgifgZ"ejF d„ « Z$ejF d„ « Z%ejF d„ « Z&ejF d„ « Z'ejF d„ « Z(ejF d „ « Z)ejT jW d d g« ejT jY d « d „ « « Z-ejT jY d!« d"„ « Z.ejT jY d#« d$„ « Z/ejT jY d%« d&„ « Z0ejT jY d'« d(„ « Z1d)„ Z2ejT jY d*« d+„ « Z3ejT jY d,« d-„ « Z4d.„ Z5d/„ Z6d0„ Z7d1„ Z8d2„ Z9ejT ju d3¬4« d5„ « Z;ejT ju d3¬4« d6„ « Z<d7„ Z=d8„ Z>d9„ Z?d:„ Z@d;„ ZAd<„ ZBd=„ ZCd>„ ZDd?„ ZEejT jW d@dAdBg« dC„ « ZFdD„ ZGdE„ ZHdF„ ZIdG„ ZJdH„ ZKdI„ ZL ejš dJ« G dK„ dL« « ZNy)Mé N)Ú assert_equal)ÚregistryÚutil)ÚENT_IOB)ÚEnglish)ÚItalian)ÚLanguage)ÚLookups)ÚEntityRecognizer)Ú
|
|||
|
|
BiluoPushDown)ÚDEFAULT_NER_MODEL)ÚDocÚSpan)ÚExampleÚ iob_to_biluoÚsplit_bilu_label©ÚVocabé )Ú make_tempdirúWho is Shaka Khan?Úentities©é é ÚPERSONzI like London and Berlin.)r é
|
|||
|
|
ÚLOC)é é r c ó — y)NÚ non_entities© r# ó ú\C:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/tests/parser/test_ner.pyÚneg_keyr& s € à r$ c ó — t « S ©Nr r# r$ r% Úvocabr) ! s € ä ‹7€Nr$ c ó — t | g d¢¬« S )N)ÚCaseyÚwentÚtoÚNewÚYorkú.©Úwords)r )r) s r% Údocr3 & s € ä ˆuÒ GÔ HÐHr$ c óz — | dd }| dd }|j |j df|j |j dfgS )Nr é é é r ÚGPE)Ú
|
|||
|
|
start_charÚend_char)r3 ÚcaseyÚnys r% Ú
|
|||
|
|
entity_annotsr= + sH € à ��!ˆH€EØ ˆQˆqˆ€Bà × Ñ ˜5Ÿ>™>¨8Ð4Ø � ‰ ˜Ÿ ™ UÐ+ð ð r$ c ód — t t | D ���cg c] \ }}}|‘Œ
|
|||
|
|
c}}}« « S c c}}}w r( )ÚsortedÚset)r= ÚsÚeÚlabels r% Ú entity_typesrD 5 s* € ä ”#±-Õ@±-¡ ! Q¨’u°-Ó@ÓAÓ BÐBùÔ@s ‘
|
|||
|
|
+c óZ — t j |¬« }t | j |« S )N©rD )r Ú get_actionsÚstrings)r) rD Úactionss r% ÚtsysrJ : s# € ä×'Ñ'°\ÔB€GÜ ˜Ÿ ™ ¨Ó 0Ð0r$ rC z
|
|||
|
|
U-JOB-NAMEi¯ c
|
|||
|
|
óü — t « }i }|j d|¬« }t j t |j
|
|||
|
|
dg¬« dgdgdgdgdg| gdœ« }d |j j |g¬
|
|||
|
|
« d v sJ ‚y ) NÚner©ÚconfigÚwordr1 r ÚtagÚdep)Úidsr2 ÚtagsÚheadsÚdepsr zJOB-NAME)Úexamplesr5 )r Ú create_piper Ú from_dictr r) ÚmovesrG )rC ÚnlprN rL Úexamples r% Útest_issue1967r\ @ sŽ € ô ‹*€CØ
|
|||
|
|
€FØ
|
|||
|
|
|
|||
|
|
�/‰/˜%¨ˆ/Ó
|
|||
|
|
/€CÜ× Ñ Ü ˆC�I‰I˜f˜XÔ&à�3Ø �XØ�GØ�SØ�GØ ˜ñ
|
|||
|
|
|
|||
|
|
ó
|
|||
|
|
€G𠘟™×.Ñ.¸¸ Ð.ÓBÀ1ÑEÑ EÐEÑ Er$ iƒ c óh — t « } | j d« }|j d« | j « t « }|j d« t |j d« j « dk( sJ ‚|j d« j } |j d || j d« j j « |j | j « « d|j d« j vsJ ‚|j d« j dk( sJ ‚y)zGTest that spurious 'extra_labels' aren't created when initializing NER.rL Ú CITIZENSHIPr Ú
|
|||
|
|
resize_outputÚ extra_labels)r^ N)r Úadd_pipeÚ add_labelÚ
|
|||
|
|
initializeÚlenÚget_pipeÚlabelsÚmodelÚattrsrY Ún_movesÚ
|
|||
|
|
from_bytesÚto_bytesÚcfg)rZ rL Únlp2rg s r% Útest_issue2179rn T sï € ô ‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€C؇M�M�-Ô Ø‡N�NÔÜ ‹9€D؇M�M�%ÔÜ ˆt�}‰}˜UÓ#×*Ñ*Ó +¨qÒ 0Ð0Ð 0Ø �M‰M˜%Ó × &Ñ &€EØ €E‡K�K�Ñ ¨¯ © °UÓ(;×(AÑ(A×(IÑ(IÔJ؇O�O�C—L‘L“NÔ#Ø § ¡ ¨uÓ!5×!9Ñ!9Ñ 9Ð9Ð 9Ø �=‰=˜Ó × &Ñ &Ð*:Ò :Ð:Ñ :r$ iQ c ó¤ — d} t | « g d¢k( sJ ‚d}t |« g d¢k( sJ ‚d}t |« g d¢k( sJ ‚d}t |« g d¢k( sJ ‚y )
|
|||
|
|
z9Test that IOB tags are correctly converted to BILUO tags.)ú B-BRAWLERú I-BRAWLERrq )rp rq z L-BRAWLER)úI-ORGrr úB-ORG)rs úL-ORGzU-ORG)úB-PERSONzI-PERSONru )ru úL-PERSONúU-PERSON)úB-MULTI-PERSONzI-MULTI-PERSONrx )rx zL-MULTI-PERSONzU-MULTI-PERSONN)r )Útags1Útags2Útags3Útags4s r% Útest_issue2385r} e sh € ð
|
|||
|
|
4€EÜ ˜Ó Ò"IÒ IÐIÐ Ià '€EÜ ˜Ó Ò"=Ò =Ð=Ð =à 0€EÜ ˜Ó Ò"FÒ FÐFÐ Fà B€EÜ ˜Ó Ò"XÒ XÐXÑ Xr$ ið
|
|||
|
|
c óâ — t « } g }|j t j | j d« dg i« g« t d« D �cg c]
|
|||
|
|
}t
|
|||
|
|
|« ‘Œ }}| j d« }t |« D ] }|j |« Œ | j « }t d« D ]6 }i }t j |« |D ] }| j |g||d¬« Œ Œ8 yc c}w ) zdTest issue that arises when too many labels are added to NER model.
|
|||
|
|
Used to cause segfault.
|
|||
|
|
z One sentencer iè rL é g à?)ÚsgdÚlossesÚdropN)r Úextendr rX Úmake_docÚrangeÚstrra Úlistrb rc ÚrandomÚshuffleÚupdate) rZ Ú
|
|||
|
|
train_dataÚirD rL Ú entity_typeÚ optimizerr� r[ s r% Útest_issue2800r� v sÖ € ô
|
|||
|
|
‹)€CØ€JØ×ÑÜ × Ñ ˜3Ÿ<™<¨Ó7¸*ÀbÐ9IÓ JÐKôô %*¨$¤KÓ0¡K˜q”C˜•F K€LÐ0Ø
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€CܘLÖ)ˆ Ø �
|
|||
|
|
‰
|
|||
|
|
�kÕ"ð *à—‘Ó €IÜ
|
|||
|
|
�2ŽYˆØˆÜ�‰�zÔ"Û!ˆGØ �J‰J˜�y i¸ÀSˆJÕ Iñ "ñ ùò 1s ÁC,i‰ c ó¨ — t « } | j d« }|j d« | j « g d¢}|j |k( sJ ‚t « }|j d« }|j
|
|||
|
|
} |j d ||j j « |j | j « « |j |k( sJ ‚y)z»Test issue that occurred in spaCy nightly where NER labels were being
|
|||
|
|
mapped to classes incorrectly after loading the model, when the labels
|
|||
|
|
were added using ner.add_label().
|
|||
|
|
rL ÚANIMAL)ÚOzB-ANIMALzI-ANIMALzL-ANIMALzU-ANIMALr_ N) r ra rb rc Ú
|
|||
|
|
move_namesrg rh rY ri rj rk )rZ rL r“ rm Úner2rg s r% Útest_issue3209r• Œ s§ € ô ‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€C؇M�M�(Ô؇N�NÔÚF€JØ �>‰>˜ZÒ 'Ð'Ð 'Ü ‹9€DØ �=‰=˜Ó €DØ �J‰J€EØ €E‡K�K�Ñ ¨¯ © ×(9Ñ(9Ô:؇O�O�C—L‘L“NÔ#Ø �?‰?˜jÒ (Ð(Ñ (r$ c óà — t « } | j d« }|j d« | j « g d¢}dh}|j |k( sJ ‚t |j « |k( sJ ‚y)zBTest that labels are inferred correctly when there's a - in label.rL z LARGE-ANIMAL)r’ zB-LARGE-ANIMALzI-LARGE-ANIMALzL-LARGE-ANIMALzU-LARGE-ANIMALN)r ra rb rc r“ r@ rf )rZ rL r“ rf s r% Útest_labels_from_BILUOr— sf € ä
|
|||
|
|
‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€C؇M�M�.Ô!؇N�NÔò€Jð Ð
|
|||
|
|
€FØ �>‰>˜ZÒ 'Ð'Ð 'Ü ˆs�z‰z‹?˜fÒ $Ð$Ñ $r$ i« c óø — t « } | j d« }|j d« | j « d| j v sJ ‚ | d« }|j d« sJ ‚|D ] }|j dk( rŒJ ‚ dddœg}| j d « }|j |« d | j v sJ ‚d| j v sJ ‚ | d« }|j d« sJ ‚|D ] }|j dk( rŒJ ‚ y
|
|||
|
|
) zDTest that running an entity_ruler after ner gives consistent resultsrL ÚPEOPLEÚhir r ÚSOFTWAREÚspacy©rC ÚpatternÚ entity_rulerN)r ra rb rc Ú
|
|||
|
|
pipe_namesÚhas_annotationÚent_iobÚ add_patterns)rZ rL Údoc1ÚtokenÚpatternsÚrulerÚdoc2s r% Útest_issue4267r© ² s € ô ‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€C؇M�M�(Ô؇N�NÔØ �C—N‘NÑ "Ð"Ð "á ˆt‹9€DØ × Ñ ˜yÔ )Ð)Ð )ÛˆØ�}‰} Ó!Ð!Ð!ð ð %°Ñ9Ð:€HØ �L‰L˜ Ó (€EØ ×Ñ�xÔ Ø ˜SŸ^™^Ñ +Ð+Ð +Ø �C—N‘NÑ "Ð"Ð "á ˆt‹9€DØ × Ñ ˜yÔ )Ð)Ð )ÛˆØ�}‰} Ó!Ð!Ð!ñ r$ iÙ c óÔ — d} d}t « }| |dœ}|j d|¬« }|j d« |j « |d« }t |j
|
|||
|
|
« dk( sJ ‚d|j
|
|||
|
|
v sJ ‚t
|
|||
|
|
|d d
|
|||
|
|
d ¬ « }t |j « |gz |_ |g}|j |d
|
|||
|
|
| |¬« t |j
|
|||
|
|
« dk( sJ ‚d |j
|
|||
|
|
v sJ ‚y)z:This should not crash or exit with some strange error codeé ç-C ëâ6?©Ú
|
|||
|
|
beam_widthÚ beam_densityÚbeam_nerrM Ú
|
|||
|
|
SOME_LABELz What do you think about Apple ?r5 r7 é ÚMY_ORG©rC ç )r‚ r® r¯ r N)
|
|||
|
|
r ra rb rc rd rf r r‡ ÚentsÚ
|
|||
|
|
beam_parse)r® r¯ rZ rN rL r3 Ú apple_entÚdocss r% Útest_issue4313rº Ì sê € ð €JØ€LÜ
|
|||
|
|
‹)€Cà Ø$ñ€Fð �,‰,�z¨&ˆ,Ó
|
|||
|
|
1€C؇M�M�,Ô Ø‡N�NÔá
|
|||
|
|
|
|||
|
|
Ð/Ó
|
|||
|
|
0€CÜ ˆs�z‰z‹?˜aÒ Ð Ð Ø ˜3Ÿ:™:Ñ %Ð%Ð %Ü�S˜!˜Q hÔ/€IÜ�C—H‘H‹~ Ñ+€C„Hð
|
|||
|
|
ˆ5€D؇N�N�4˜c¨jÀ|€NÔTÜ ˆs�z‰z‹?˜aÒ Ð Ð Ø �s—z‘zÑ !Ð!Ñ !r$ c ó² — t j |d|i« }| j |d¬« }|D �cg c] }| j |« ‘Œ }}|g d¢k( sJ ‚y c c}w )Nr F)Ú_debug)rw r’ r’ úB-GPEúL-GPEr’ )r rX Úget_oracle_sequenceÚget_class_name)rJ r3 r= r[ Ú act_classesÚactÚnamess r% Útest_get_oracle_movesrÄ ç sa € Ü× Ñ j°-Ð%@ÓA€GØ×*Ñ*¨7¸5Ð*ÓA€KÙ1<Ó =±¨#ˆT×
|
|||
|
|
Ñ
|
|||
|
|
Õ
|
|||
|
|
%°€EÐ =Ø ÒAÒ AÐAÑ Aùò
|
|||
|
|
>s °Ac ó¶ — || j d< t |ddg¬« }ddg}t j |d|i« }t |j
|
|||
|
|
ddd ¬
|
|||
|
|
« t |j
|
|||
|
|
dd d ¬
|
|||
|
|
« g|j
|
|||
|
|
j |< | j |« }|D �cg c] }| j |« ‘Œ }}|sJ ‚|d d k7 sJ ‚|d d
|
|||
|
|
k7 sJ ‚|d dk7 sJ ‚yc c}w )z›Test that we don't get stuck in a two word input when we have a negative
|
|||
|
|
span. This could happen if we don't have the right check on the B action.
|
|||
|
|
r& ÚAÚBr1 Nr r r5 r’ r´ r r ru rv © rl r r rX r ÚyÚspansr¿ rÀ © rJ r) r& r3 r= r[ rÁ r rà s r% Ú$test_negative_samples_two_word_inputrÌ î sí € ð "€D‡H�HˆYÑÜ
|
|||
|
|
|
|||
|
|
ˆe˜C ˜:Ô
|
|||
|
|
&€Cؘ4�L€MÜ× Ñ j°-Ð%@ÓA€Gô
|
|||
|
|
ˆW�Y‰Y˜˜1 CÔ(Ü ˆW�Y‰Y˜˜1 HÔ-ð €G‡I�I‡O�O�GÑ ð ×*Ñ*¨7Ó3€KÙ1<Ó =±¨#ˆT×
|
|||
|
|
Ñ
|
|||
|
|
Õ
|
|||
|
|
%°€EÐ =Ù €Lˆ5Ø �‰8�sŠ?Ј?Ø �‰8�zÒ !Ð!Ð !Ø �‰8�zÒ !Ð!Ñ !ùò
|
|||
|
|
>s ÂCc ó¢ — || j d< t |g d¢¬« }g d¢}t j |d|i« }t |j
|
|||
|
|
ddd¬ « t |j
|
|||
|
|
dd
|
|||
|
|
d ¬ « g|j
|
|||
|
|
j |< | j |« }|D �cg c] }| j |« ‘Œ }}|sJ ‚|d dk7 sJ ‚|d d k7 sJ ‚y
|
|||
|
|
c c}w )úHTest that we exclude a 2-word entity correctly using a negative example.r& )rÆ rÇ ÚCr1 )NNNr r r5 r’ r´ r r ru NrÈ rË s r% Ú&test_negative_samples_three_word_inputrÐ sÑ € à!€D‡H�HˆYÑÜ
|
|||
|
|
|
|||
|
|
ˆeš?Ô
|
|||
|
|
+€CÚ&€MÜ× Ñ j°-Ð%@ÓA€Gô
|
|||
|
|
ˆW�Y‰Y˜˜1 CÔ(Ü ˆW�Y‰Y˜˜1 HÔ-ð €G‡I�I‡O�O�GÑ ð ×*Ñ*¨7Ó3€KÙ1<Ó =±¨#ˆT×
|
|||
|
|
Ñ
|
|||
|
|
Õ
|
|||
|
|
%°€EÐ =Ù €Lˆ5Ø �‰8�sŠ?Ј?Ø �‰8�zÒ !Ð!Ñ !ùò
|
|||
|
|
>s ÂC c óž — || j d< t |dg¬« }dg}t j |d|i« }t |j
|
|||
|
|
ddd¬ « t |j
|
|||
|
|
ddd
|
|||
|
|
¬ « g|j
|
|||
|
|
j |< | j |« }|D �cg c] }| j |« ‘Œ }}|sJ ‚|d dk7 sJ ‚|d d k7 sJ ‚yc c}w ) rÎ r& rÆ r1 Nr r r5 r’ r´ r rw rÈ rË s r% Ú test_negative_samples_U_entityrÒ sÕ € à!€D‡H�HˆYÑÜ
|
|||
|
|
|
|||
|
|
ˆe˜C˜5Ô
|
|||
|
|
!€CØ�F€MÜ× Ñ j°-Ð%@ÓA€Gô
|
|||
|
|
ˆW�Y‰Y˜˜1 CÔ(Ü ˆW�Y‰Y˜˜1 HÔ-ð €G‡I�I‡O�O�GÑ ð ×*Ñ*¨7Ó3€KÙ1<Ó =±¨#ˆT×
|
|||
|
|
Ñ
|
|||
|
|
Õ
|
|||
|
|
%°€EÐ =Ù €Lˆ5Ø �‰8�sŠ?Ј?Ø �‰8�zÒ !Ð!Ñ !ùò
|
|||
|
|
>s ÂC
|
|||
|
|
c óˆ — t j |¬« }t | j |d¬« }|j d dk( sJ ‚y )NrF r" )Úincorrect_spans_keyr& )r rG rH rl )r) rD rI rJ s r% Ú%test_negative_sample_key_is_in_configrÕ * s; € Ü×'Ñ'°\ÔB€GÜ ˜Ÿ ™ ¨À^Ô T€DØ �8‰8�IÑ .Ò 0Ð0Ñ 0r$ zNo longer supported)Úreasonc ó6 — g d¢}g d¢}t | |¬« }t j |||dœ« }t | j « }d}|D ]¾ }|€Œ|dk( r"|j |j
|
|||
|
|
d« d« Œ-t |« \ }} |j |j
|
|||
|
|
d« | « |j |j
|
|||
|
|
d « | « |j |j
|
|||
|
|
d
|
|||
|
|
« | « |j |j
|
|||
|
|
d « | « ŒÀ |j |« y ) N)rÇ Ú52ÚBomber)NNz L-PRODUCTr1 )r2 r ©ÚMrÇ ÚIÚLÚUr’ r’ Ú rÇ rÜ rÝ rÞ © r r rX r rH Ú
|
|||
|
|
add_actionÚindexr r¿ ©
|
|||
|
|
Úen_vocabr2 Ú
|
|||
|
|
biluo_tagsr3 r[ rY Ú
|
|||
|
|
move_typesrP ÚactionrC s
|
|||
|
|
r% Útest_oracle_moves_missing_Brè 2 s € â !€EÚ*€Jä
|
|||
|
|
|
|||
|
|
ˆh˜eÔ
|
|||
|
|
$€CÜ× Ñ ¨uÀ*Ñ%MÓN€Gä ˜(×*Ñ*Ó +€EØ/€JÛˆØ ˆ;Ø Ø
|
|||
|
|
�CŠZØ × Ñ ˜Z× -Ñ -¨cÓ 2°BÕ 7ä ,¨SÓ 1‰MˆF�EØ × Ñ ˜Z× -Ñ -¨cÓ 2°EÔ :Ø × Ñ ˜Z× -Ñ -¨cÓ 2°EÔ :Ø × Ñ ˜Z× -Ñ -¨cÓ 2°EÔ :Ø × Ñ ˜Z× -Ñ -¨cÓ 2°EÕ :ð ð
|
|||
|
|
× Ñ ˜gÕ&r$ c ón — g d¢}g d¢}t | |¬« }t j |d|i« }t | j « }d}|D ][ }|€Œ|dk( r"|j |j
|
|||
|
|
d« d« Œ-t |« \ }} |j |j
|
|||
|
|
|« | « Œ] |j |« y )N) Ú
|
|||
|
|
productionú
|
|||
|
|
ÚofÚNorthroprë zCorp.rë z'sÚradar) r’ r’ r’ rs Nrr rt r’ r’ r1 r rÚ r’ rß rà rã s
|
|||
|
|
r% Ú test_oracle_moves_whitespacerï L s± € â V€EÚK€Jä
|
|||
|
|
|
|||
|
|
ˆh˜eÔ
|
|||
|
|
$€CÜ× Ñ j°*Ð%=Ó>€Gä ˜(×*Ñ*Ó +€EØ/€JÛˆØ ˆ;Ø Ø
|
|||
|
|
�CŠZØ × Ñ ˜Z× -Ñ -¨cÓ 2°BÕ 7ä ,¨SÓ 1‰MˆF�EØ × Ñ ˜Z× -Ñ -¨fÓ 5°uÕ =ð ð
|
|||
|
|
× Ñ ˜gÕ&r$ c ó — t « } | d« }i }| j d|¬« }|D �cg c] }|j ‘Œ c}g d¢k( sJ ‚|D �cg c] }|j ‘Œ c}g d¢k( sJ ‚|j j dd« |j
|
|||
|
|
d« |j j |g« d }|j j |d « |j j |d « |j j |d « |j j |d
|
|||
|
|
« sJ ‚t « } |d« }i }|j d|¬« }|j g |d d gd ¬
|
|||
|
|
« |D �cg c] }|j ‘Œ c}g d¢k( sJ ‚|D �cg c] }|j ‘Œ c}g d¢k( sJ ‚|j j dd« |j j dd« |j
|
|||
|
|
d« |j j |g« d } |j j | d « |j j | d « |j j | d « |j j | d
|
|||
|
|
« rJ ‚|j j | d« sJ ‚|j j | d« |j j | d
|
|||
|
|
« rJ ‚|j j | d« sJ ‚yc c}w c c}w c c}w c c}w )z5Test succesful blocking of tokens to be in an entity.úI live in New YorkrL rM ©rß rß rß rß rß r7 rß r8 r r’ r½ r6 Ú
|
|||
|
|
unmodified©ÚblockedÚdefault)rß rß rß rÇ rÇ é zU-N) r rW Úent_iob_Ú ent_type_rY rá rb Ú
|
|||
|
|
init_batchÚapply_transitionÚis_validÚset_ents)
|
|||
|
|
Únlp1r¤ rN Úner1r¥ Ústate1rm r¨ r” Ústate2s
|
|||
|
|
r% Útest_accept_blocked_tokenr a s¯ € ô ‹9€DÙ Ð$Ó %€DØ
|
|||
|
|
€FØ × Ñ ˜E¨&Ð Ó 1€DÙ(,Ó -©˜uˆE�N‹N¨Ñ -Ò1EÒ EÐEÐ EÙ)-Ó .© ˆE�O‹O¨Ñ .Ò2FÒ FÐFÐ Fð ‡J�J×ј!˜RÔ Ø‡N�N�5Ôà
|
|||
|
|
�Z‰Z×
|
|||
|
|
"Ñ
|
|||
|
|
" D 6Ó
|
|||
|
|
*¨1Ñ
|
|||
|
|
-€F؇J�J× Ñ ¨Ô,؇J�J× Ñ ¨Ô,؇J�J× Ñ ¨Ô,à �:‰:× Ñ ˜v wÔ /Ð/Ð /ô ‹9€DÙ Ð$Ó %€DØ
|
|||
|
|
€FØ × Ñ ˜E¨&Ð Ó 1€Dð ‡M�M�"˜t A a˜y˜k°<€MÔ@Ù(,Ó -©˜uˆE�N‹N¨Ñ -Ò1GÒ GÐGÐ GÙ)-Ó .© ˆE�O‹O¨Ñ .Ò2FÒ FÐFÐ Fð ‡J�J×ј!˜RÔ Ø‡J�J×ј!˜RÔ Ø‡N�N�5ÔØ
|
|||
|
|
�Z‰Z×
|
|||
|
|
"Ñ
|
|||
|
|
" D 6Ó
|
|||
|
|
*¨1Ñ
|
|||
|
|
-€F؇J�J× Ñ ¨Ô,؇J�J× Ñ ¨Ô,؇J�J× Ñ ¨Ô,à�z‰z×"Ñ" 6¨7Ô3Ð3Ð 3Ø �:‰:× Ñ ˜v tÔ ,Ð,Ð ,؇J�J× Ñ ¨Ô-à�z‰z×"Ñ" 6¨7Ô3Ð3Ð 3Ø �:‰:× Ñ ˜v tÔ ,Ð,Ñ ,ùòO .ùÚ .ùò* .ùÚ .s ¬K:Á
|
|||
|
|
K?Å+LÆ L c ó¦ — dddgifddg ifg} t « }g }| D ]< }|j t j |j |d « |d « « Œ> |j dd¬ « }|j
|
|||
|
|
d
|
|||
|
|
« |j « t d « D ]5 }i }t j |d ¬
|
|||
|
|
« }|D ] }|j ||¬« Œ Œ7 y)z7Test that training an empty text does not throw errors.r r r rß r r5 rL T©Úlastr r é ©Úsize©r� N) r Úappendr rX r„ ra rb rc r… r Ú minibatchrŠ © r‹ rZ Útrain_examplesÚtrL Úitnr� ÚbatchesÚbatchs r% Útest_train_emptyr ’ sÒ € ð
|
|||
|
|
|
|||
|
|
Ð->Ð,?Ð @ÐAØ ˆj˜"Ð
|
|||
|
|
Ð ð€Jô
|
|||
|
|
‹)€CØ€NÛ
|
|||
|
|
ˆØ× Ñ œg× /Ñ /°· ± ¸Q¸q¹TÓ0BÀAÀaÁDÓ IÕJð à
|
|||
|
|
|
|||
|
|
�,‰,�u 4ˆ,Ó
|
|||
|
|
(€C؇M�M�(Ô؇N�NÔÜ�QŽxˆØˆÜ—.‘. °aÔ8ˆÛ ˆEØ �J‰J�u VˆJÕ ,ñ ñ r$ c óø — dddgifg} t « }g }| D ]< }|j t j |j |d « |d « « Œ> |j dd¬« }|j
|
|||
|
|
d « |j « t d
|
|||
|
|
« D ]W }i }t j |d ¬ « }|D ]7 }t j t « 5 |j ||¬
|
|||
|
|
« ddd« Œ9 ŒY y# 1 sw Y ŒFxY w)zFTest that the deprecated negative entity format raises a custom error.r r )r r z!PERSONr r5 rL Tr r r r r r N)r r
|
|||
|
|
r rX r„ ra rb rc r… r r ÚpytestÚraisesÚ
|
|||
|
|
ValueErrorrŠ r s r% Ú test_train_negative_deprecatedr § sâ € ð
|
|||
|
|
|
|||
|
|
Ð-?Ð,@Ð AÐBð€Jô ‹)€CØ€NÛ
|
|||
|
|
ˆØ× Ñ œg× /Ñ /°· ± ¸Q¸q¹TÓ0BÀAÀaÁDÓ IÕJð à
|
|||
|
|
|
|||
|
|
�,‰,�u 4ˆ,Ó
|
|||
|
|
(€C؇M�M�(Ô؇N�NÔÜ�QŽxˆØˆÜ—.‘. °aÔ8ˆÛ ˆEÜ— ‘ œzÕ*Ø—
|
|||
|
|
‘
|
|||
|
|
˜5¨�
|
|||
|
|
Ô0÷ +Ð*ñ ñ ÷ +Ð*ús ÃC0Ã0C9 c óì — t « } | j d« | j « | d« }|D �cg c] }|j ‘Œ c}g d¢k( sJ ‚|D �cg c] }|j ‘Œ c}g d¢k( sJ ‚i }| j d|¬« }|j j dd« |j d« |j j |g« d }|j j |d
|
|||
|
|
« sJ ‚|j j |d « sJ ‚|j j |d
|
|||
|
|
« |j j |d « sJ ‚|j j |d
|
|||
|
|
« sJ ‚y c c}w c c}w )NrL rñ )r’ r’ r’ r’ r’ rò rM r7 rß r8 r r½ zU-GPEzI-GPEr¾ ) r ra rc rø rù rW rY rá rb rú rü rû )rZ r3 r¥ rN r” Ústates r% Útest_overwrite_tokenr ¼ sD € Ü
|
|||
|
|
‹)€C؇L�L�Ô؇N�NÔá
|
|||
|
|
|
|||
|
|
Ð"Ó
|
|||
|
|
#€CÙ(+Ó ,©˜uˆE�N‹N¨Ñ ,Ò0IÒ IÐIÐ IÙ),Ó -© ˆE�O‹O¨Ñ -Ò1EÒ EÐEÐ Eà
|
|||
|
|
€FØ �?‰?˜5¨ˆ?Ó 0€D؇J�J×ј!˜RÔ Ø‡N�N�5ÔØ �J‰J× !Ñ ! 3 %Ó (¨Ñ +€EØ �:‰:× Ñ ˜u gÔ .Ð.Ð .Ø �:‰:× Ñ ˜u gÔ .Ð.Ð .؇J�J× Ñ wÔ/Ø �:‰:× Ñ ˜u gÔ .Ð.Ð .Ø �:‰:× Ñ ˜u gÔ .Ð.Ñ .ùò -ùÚ -s ¸E,ÁE1c óÜ — t « } | j d« }|j d« | j « | d« }g d¢}|D �cg c] }|j ‘Œ c}|k( sJ ‚y c c}w )NrL ÚMY_LABELz3John is watching the news about Croatia's elections) r’ r’ r’ r’ r’ r’ r’ r’ r’ )r ra rb rc rø )rZ rL r3 Úresultr¥ s r% Útest_empty_nerr Ñ sa € Ü
|
|||
|
|
‹)€CØ
|
|||
|
|
|
|||
|
|
�,‰,�uÓ
|
|||
|
|
€C؇M�M�*Ô Ø‡N�NÔÙ
|
|||
|
|
|
|||
|
|
ÐCÓ
|
|||
|
|
D€Câ
|
|||
|
|
:€FÙ(+Ó ,©˜uˆE�N‹N¨Ñ ,°Ò 6Ð6Ñ 6ùÒ ,s Á
|
|||
|
|
A)c ó| — t « } dddœg}| j d« }| j d« }|j d« | j « |j |« | d« }g d¢}g d ¢}|D �cg c] }|j
|
|||
|
|
‘Œ c}|k( sJ ‚|D �cg c] }|j ‘Œ c}|k( sJ ‚y
|
|||
|
|
c c}w c c}w ) zLTest that an NER works after an entity_ruler: the second can add annotationsÚTHINGÚThisr� rŸ rL r ú*This is Antti Korhonen speaking in Finland©rÇ r’ r’ r’ r’ r’ r’ ©r rß rß rß rß rß rß N©r ra rb rc r£ rø rù )rZ r¦ r§ Ú
|
|||
|
|
untrained_nerr3 Ú
|
|||
|
|
expected_iobsÚexpected_typesr¥ s r% Útest_ruler_before_nerr) Ü s» € ä
|
|||
|
|
‹)€Cð "¨fÑ5Ð6€HØ �L‰L˜ Ó (€Eð —L‘L Ó'€MØ×јJÔ'؇N�NÔØ ×Ñ�xÔ Ù
|
|||
|
|
|
|||
|
|
Ð:Ó
|
|||
|
|
;€CÚ7€MÚ6€NÙ(+Ó ,©˜uˆE�N‹N¨Ñ ,°
|
|||
|
|
Ò =Ð=Ð =Ù),Ó -© ˆE�O‹O¨Ñ -° Ò ?Ð?Ñ ?ùò -ùÚ -s Á9B4ÂB9c ó‚ — ddi}dt i}t j |d¬« d }t | |fi |¤Ž t | |« y )NÚupdate_with_oracle_cut_sizeéd rg T)Úvalidate)r
|
|||
|
|
r Úresolver )rä rN rl rg s r% Útest_ner_constructorr/ ð sK € à% sð€Fð Ô%Ð
|
|||
|
|
&€CÜ × Ñ ˜S¨4Ô 0°Ñ 9€EÜ�X˜uÑ/¨Ò/Ü�X˜uÕ%r$ c ó€ — t « } | j dd¬« }|j d« | j « dddœg}| j d« }|j |« | d « }g d
|
|||
|
|
¢}g d ¢}|D �cg c] }|j
|
|||
|
|
‘Œ c}|k( sJ ‚|D �cg c] }|j ‘Œ c}|k( sJ ‚y c c}w c c}w )
|
|||
|
|
zTTest that an entity_ruler works after an NER: the second can overwrite O annotationsrL Úuner)Únamer r r! r� rŸ r" r# r$ Nr% )rZ r& r¦ r§ r3 r' r( r¥ s r% Útest_ner_before_rulerr3 ú s¿ € ä
|
|||
|
|
‹)€Cð —L‘L ¨V�LÓ4€MØ×јJÔ'؇N�NÔð "¨fÑ5Ð6€HØ �L‰L˜ Ó (€EØ ×Ñ�xÔ á
|
|||
|
|
|
|||
|
|
Ð:Ó
|
|||
|
|
;€CÚ7€MÚ6€NÙ(+Ó ,©˜uˆE�N‹N¨Ñ ,°
|
|||
|
|
Ò =Ð=Ð =Ù),Ó -© ˆE�O‹O¨Ñ -° Ò ?Ð?Ñ ?ùò -ùÚ -s Á;B6ÂB;c óX — t « } | j ddddœ¬« | j d« }|j d« | j « | d« }g d ¢}g d
|
|||
|
|
¢}|D �cg c] }|j ‘Œ c}|k( sJ ‚|D �cg c] }|j
|
|||
|
|
‘Œ c}|k( sJ ‚y c c}w c c}w ) zITest functionality for blocking tokens so they can't be in a named entityÚblockerr r7 )ÚstartÚendrM rL r z,This is Antti L Korhonen speaking in Finland)r’ r’ rÇ rÇ rÇ r’ r’ r’ )rß rß rß rß rß rß rß rß N)r ra rb rc rø rù )rZ r& r3 r' r( r¥ s r% Útest_block_nerr8 s¥ € ô ‹)€C؇L�L�¨Q°qÑ#9€LÔ:Ø—L‘L Ó'€MØ×јJÔ'؇N�NÔÙ
|
|||
|
|
|
|||
|
|
Ð<Ó
|
|||
|
|
=€CÚ<€MÚ5€NÙ(+Ó ,©˜uˆE�N‹N¨Ñ ,°
|