274 lines
44 KiB
Plaintext
274 lines
44 KiB
Plaintext
|
Ë
|
|||
|
|
?û gþž ã # ó” — d dl Z d dlZd dlZd dlmZ d dlmZmZm Z m
|
|||
|
|
Z
|
|||
|
|
d dl m Z d dl
|
|||
|
|
Z
|
|||
|
|
d dl
|
|||
|
|
mZ d dlmZmZ d dlmZ d dlmZ d d lmZ d d
|
|||
|
|
lmZmZmZ d d l m Z m Z m Z d d l m!Z! d d
|
|||
|
|
l"m#Z# d dl$m%Z%m&Z& d dl'm(Z( d dl)m*Z* ddl+m,Z- ddlm.Z. dddddœifdddddœifgZ/ddddddœifd dddddœifgZ0d Z1 e« je e1« d Z3d „ Z4d „ Z5ejl jo d!« d"„ « Z8ejl jo d#« d$„ « Z9ejl ju d%eeee e e g« ejl jo d&« d'„ « « Z;d(Z<ejl ju d)d*d+g« ejl jo d,« d-„ « « Z=ejl jo d.« d/„ « Z>ejl jo d0« d1„ « Z?ejl j� d2¬3« d4„ « ZAejl j� d2¬3« d5„ « ZBejl ju d6d*d+g« d7„ « ZCejl ju d8d*e4fd+e5fg« d9„ « ZDejl ju d6d*d+g« d:„ « ZEejl ju d8d*e4fd+e5fg« d;„ « ZFejl jŽ ejl ju d<d*d=d>d?d@dAœfd*d=d>d>d@dAœfd+d=d?d?d@dAœfd+d=d?d>d@dAœfd*dBd?ddCdDddEdEddFœ fd+dBd?ddCdDddEdEddFœ fd*dGe!d=d>d?d@dAœdHœfd*dGe!d=d>d>d@dAœdHœfd+dGe!d=d?d?d@dAœdHœfd+dGe!d=d?d>d@dAœdHœfd*dIe!d>dJœfd+dIe!d?dJœfg « dK„ « « ZHejl ju d<d*dLd>d?d@dAœfd*dLd>d>d@dAœfd+dLd?d?d@dAœfd+dLd?d>d@dAœfd*dMe!d>d>d>d>d>dNœfd+dMe!d?d>d>d>d>dNœfg« dO„ « ZIejl ju d<d*dLd>d?d@dAœfd*dLd>d>d@dAœfd+dLd?d?d@dAœfd+dLd?d>d@dAœfd*dMe!d>d>d>d>d>dNœfd+dMe!d?d>d>d>d>dNœfg« dP„ « ZJdQ„ ZKejl ju d<d+dGe3dLd?dEd?dRœdHœfd*dGe3dLd>dSd?dRœdHœfd*dTe3d>dJœfd+dTe3d?dJœfd*dMe3d>d>d>d>d>dNœfd+dMe3d?d>d>d>d>dNœfg« dU„ « ZLejl ju dVd*e4e/fd+e5e0fg« dW„ « ZMdX„ ZNdY„ ZOejl jŽ ejl ju dZd+e0d=d?dEd?dRœfd*e/d=d>d[d?dRœfd+e0dBd?ddCdDddEdEddFœ fd*e/dBd?ddCdDddEdEddFœ fd*e/dIe!d>dJœfd+e0dIe!d?dJœfd+e0d\d?dEd?dRœfd*e/d\d>d[d?dRœfd+e0d\d?d@d>dRœfd*e/d\d>dd>dRœfd+e0dLd?dEd?dRœfd*e/dLd>d[d?dRœfd+e0dLd?d@d>dRœfd*e/dLd>dd>dRœfd+e0dGe!dLd?dEd?dRœdHœfd*e/dGe!dLd>dSd?dRœdHœfd*e/d]e!d>dJœfd+e0d]e!d?dJœfd*e/dTe!d>dJœfd+e0dTe!d?dJœfd*e/dMe!d>d>d>d>d>dNœfd+e0dMe!d?d>d>d>d>dNœfg« d^„ « « ZPd_„ ZQd`„ ZRda„ ZSdb„ ZTejl ju dcdddeg« dfeUdgeVfdh„« ZWejl ju didjdkg« dfeUdleVfdm„« ZXdn„ ZYdo„ ZZejl ju dpdqdrg« ds„ « Z[y)té N)Úassert_almost_equal)ÚConfigÚ compoundingÚfix_random_seedÚget_current_ops)Úmsg)Úutil)Úprint_prf_per_typeÚprint_textcats_auc_per_cat)ÚEnglish)ÚLanguage)ÚTextCategorizer)Úsingle_label_bow_configÚsingle_label_cnn_configÚsingle_label_default_config)Úmulti_label_bow_configÚmulti_label_cnn_configÚmulti_label_default_config)ÚDEFAULT_TOK2VEC_MODEL)ÚScorer)ÚDocÚDocBin)ÚExample)Úinit_nlpé )Úbuild_lazy_init_tok2vec)Ú make_tempdirz
|
|||
|
|
I'm so happy.Úcatsç ð?ç )ÚPOSITIVEÚNEGATIVEz I'm so angryzI'm angry and confused)ÚANGRYÚCONFUSEDÚHAPPYzI'm confused but happyz?
|
|||
|
|
[model]
|
|||
|
|
@architectures = "test.LazyInitTok2Vec.v1"
|
|||
|
|
width = 96
|
|||
|
|
Úmodelc ó ‡— g Št D ]< }‰j t j | j |d « |d « « Œ> ˆfd„}|S )Nr é c ó •— ‰ S ©N© ©Útrain_exampless €úbC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/tests/pipeline/test_textcat.pyÚ get_examplesz4make_get_examples_single_label.<locals>.get_examples: ó ø€ Ø Ð ó )ÚTRAIN_DATA_SINGLE_LABELÚappendr Ú from_dictÚmake_doc©ÚnlpÚtr/ r- s @r. Ú make_get_examples_single_labelr9 5 sK ø€ Ø€Nß
|
|||
|
|
$ˆØ× Ñ œg× /Ñ /°· ± ¸Q¸q¹TÓ0BÀAÀaÁDÓ IÕJð %ô ð Ðr1 c ó ‡— g Št D ]< }‰j t j | j |d « |d « « Œ> ˆfd„}|S )Nr r( c ó •— ‰ S r* r+ r, s €r. r/ z3make_get_examples_multi_label.<locals>.get_examplesE r0 r1 )ÚTRAIN_DATA_MULTI_LABELr3 r r4 r5 r6 s @r. Ú make_get_examples_multi_labelr= @ sK ø€ Ø€Nß
|
|||
|
|
#ˆØ× Ñ œg× /Ñ /°· ± ¸Q¸q¹TÓ0BÀAÀaÁDÓ IÕJð $ô ð Ðr1 i c
|
|||
|
|
óˆ — ddg} g d¢}g d¢}t j d« }g }t ||« D ]N \ }}| D �ci c] }|||k( “Œ
|
|||
|
|
}}|j t j
|
|||
|
|
|j
|
|||
|
|
|« d|i« « ŒP ddd d
|
|||
|
|
d œ} |j d d
|
|||
|
|
| id¬« }
|
|||
|
|
| D ] }|
|
|||
|
|
j |« Œ |j d ¬« 5 |j « } t d« D ]B } i }
|
|||
|
|
t j |t ddd« ¬« }|D ] }|j || d|
|
|||
|
|
¬« Œ ŒD ddd« yc c}w # 1 sw Y yxY w)zXTest whether adding n-grams in the textcat works even when n > token length of some docsÚ offensiveÚ inoffensive©zThis is an offensive textz!This is the second offensive textÚinoff©r? r? r@ Úenr úspacy.TextCatBOW.v1Tr F©ú@architecturesÚexclusive_classesÚ
|
|||
|
|
ngram_sizeÚno_output_layerÚtextcatr& ©ÚconfigÚlast©Úenableé ç @ç @@çj¼t“ð?©Úsizeçš™™™™™¹?©ÚexamplesÚsgdÚdropÚlossesN)ÚspacyÚblankÚzipr3 r r4 r5 Úadd_pipeÚ add_labelÚ select_pipesÚ
|
|||
|
|
initializeÚranger Ú minibatchr Úupdate)Úunique_classesÚx_trainÚy_trainr7 Ú
|
|||
|
|
train_dataÚtextÚtrain_instanceÚlabelÚcat_dictr& rK Ú optimizerÚir\ ÚbatchesÚbatchs r. Útest_issue3611rs K sT € ð " =Ð1€Nò€Gò
|
|||
|
|
8€GÜ
|
|||
|
|
�+‰+�dÓ
|
|||
|
|
€Cà€JÜ # G¨WÖ 5Ñ ˆˆnÙ@NÓOÁ°u�E˜5 NÑ2Ñ2ÀˆÐOØ×Ñœ'×+Ñ+¨C¯L©L¸Ó,>ÀÈÐ@RÓSÕTð !6ð
|
|||
|
|
0Ø !ØØ ñ
|
|||
|
|
€Eð �l‰l˜9¨g°uÐ-=ÀDˆlÓI€GÛ ˆØ×ј%Õ ð ð
|
|||
|
|
|
|||
|
|
× Ñ Ð Õ +Ø—N‘NÓ$ˆ Ü�q–ˆA؈FÜ—n‘n Z´kÀ#ÀtÈUÓ6SÔTˆGã �Ø—
|
|||
|
|
‘
|
|||
|
|
E¨y¸sÈ6�
|
|||
|
|
ÕRñ !ñ ÷
|
|||
|
|
,Ð +ùò P÷
|
|||
|
|
,Ð +ús º
|
|||
|
|
D3ÃA!D8Ä8Ei¾ c
|
|||
|
|
óè — ddg} g d¢}g d¢}t j d« }g }t ||« D ]N \ }}| D �ci c] }|||k( “Œ
|
|||
|
|
}}|j t j
|
|||
|
|
|j
|
|||
|
|
|« d|i« « ŒP ddd d
|
|||
|
|
d œ} |j d d
|
|||
|
|
| id¬« }
|
|||
|
|
| D ] }|
|
|||
|
|
j |« Œ |j d ¬« 5 |j « } t d« D ]B } i }
|
|||
|
|
t j |t ddd« ¬« }|D ] }|j || d|
|
|||
|
|
¬« Œ ŒD ddd« |d« }|j d dk( sJ ‚|j d dk( sJ ‚yc c}w # 1 sw Y Œ?xY w)z.Test whether textcat works fine with empty docr? r@ rA rC rD r rE Tr FrF rK r& rL rO rQ rR rS rT rU rW rX NÚ r )r] r^ r_ r3 r r4 r5 r` ra rb rc rd r re r rf r )rg rh ri r7 rj rk rl rm rn r& rK ro rp r\ rq rr Údocs r. Útest_issue4030rw p sŠ € ð " =Ð1€Nò€Gò
|
|||
|
|
8€GÜ
|
|||
|
|
�+‰+�dÓ
|
|||
|
|
€Cà€JÜ # G¨WÖ 5Ñ ˆˆnÙ@NÓOÁ°u�E˜5 NÑ2Ñ2ÀˆÐOØ×Ñœ'×+Ñ+¨C¯L©L¸Ó,>ÀÈÐ@RÓSÕTð !6ð
|
|||
|
|
0Ø !ØØ ñ
|
|||
|
|
€Eð �l‰l˜9¨g°uÐ-=ÀDˆlÓI€GÛ ˆØ×ј%Õ ð ð
|
|||
|
|
|
|||
|
|
× Ñ Ð Õ +Ø—N‘NÓ$ˆ Ü�q–ˆA؈FÜ—n‘n Z´kÀ#ÀtÈUÓ6SÔTˆGã �Ø—
|
|||
|
|
‘
|
|||
|
|
E¨y¸sÈ6�
|
|||
|
|
ÕRñ !ñ ÷
|
|||
|
|
,ñ ˆb‹'€CØ �8‰8�KÑ CÒ 'Ð'Ð 'Ø �8‰8�MÑ " cÒ )Ð)Ñ )ùò1 P÷
|
|||
|
|
,Ð +ús º
|
|||
|
|
E#ÃA!E(Å(E1Útextcat_configi¯ c óö — d}t « j | « }g }t d« D ]Ð }t d« t « }d}dddddœi}|j ||d ¬
|
|||
|
|
« }t
|
|||
|
|
|d « D ] } |j | « Œ |j « |j |« }
|
|||
|
|
|j t j |
|
|||
|
|
|« g« |j j |
|
|||
|
|
g« } |j | d « ŒÒ t! |« dk( sJ ‚t# « } t% | j' |d « | j' |d « d ¬
|
|||
|
|
« t% | j' |d « | j' |d « d ¬
|
|||
|
|
« y)zWTest that after fixing the random seed, the results of the pipeline are truly identicalrK rQ r zUOnce hot, form ping-pong-ball-sized balls of the mixture, each weighing roughly 25 g.r r r )ÚLabe1ÚLabel2ÚLabel3TrL r( é ©Údecimalr N)r Úfrom_strrd r r r` Úsetra rc r5 rf r r4 r& Úpredictr3 Úlenr r Úto_numpy)
|
|||
|
|
rx Ú componentÚpipe_cfgÚresultsrp r7 rk ÚannotsÚpiperm rv ÚresultÚopss
|
|||
|
|
r. Útest_issue5551rŒ ™ sL € ð €Iä‹x× Ñ Ó0€HØ€GÜ
|
|||
|
|
�1ŽXˆÜ˜ÔÜ‹iˆØfˆØ C°3À#ÑFÐGˆØ�|‰|˜I¨h¸Tˆ|ÓBˆÜ˜ ™ Ö(ˆEØ �N‰N˜5Õ !ð )ð �‰ÔØ�l‰l˜4Ó ˆØ �
|
|||
|
|
‰
|
|||
|
|
”G×%Ñ% c¨6Ó2Ð3Ô4à—‘×#Ñ# S EÓ*ˆØ�‰�v˜a‘yÕ!ð ô ˆw‹<˜1Ò Ð Ð Ü
|
|||
|
|
Ó
|
|||
|
|
€CܘŸ ™ W¨Q¡ZÓ0°#·,±,¸wÀq¹zÓ2JÐTUÕVܘŸ ™ W¨Q¡ZÓ0°#·,±,¸wÀq¹zÓ2JÐTUÖVr1 aÜ
|
|||
|
|
[paths]
|
|||
|
|
train = "TRAIN_PLACEHOLDER"
|
|||
|
|
raw = null
|
|||
|
|
init_tok2vec = null
|
|||
|
|
vectors = null
|
|||
|
|
|
|||
|
|
[system]
|
|||
|
|
seed = 0
|
|||
|
|
gpu_allocator = null
|
|||
|
|
|
|||
|
|
[nlp]
|
|||
|
|
lang = "en"
|
|||
|
|
pipeline = ["textcat"]
|
|||
|
|
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}
|
|||
|
|
disabled = []
|
|||
|
|
before_creation = null
|
|||
|
|
after_creation = null
|
|||
|
|
after_pipeline_creation = null
|
|||
|
|
batch_size = 1000
|
|||
|
|
|
|||
|
|
[components]
|
|||
|
|
|
|||
|
|
[components.textcat]
|
|||
|
|
factory = "TEXTCAT_PLACEHOLDER"
|
|||
|
|
|
|||
|
|
[corpora]
|
|||
|
|
|
|||
|
|
[corpora.train]
|
|||
|
|
@readers = "spacy.Corpus.v1"
|
|||
|
|
path = ${paths:train}
|
|||
|
|
|
|||
|
|
[corpora.dev]
|
|||
|
|
@readers = "spacy.Corpus.v1"
|
|||
|
|
path = ${paths:train}
|
|||
|
|
|
|||
|
|
|
|||
|
|
[training]
|
|||
|
|
train_corpus = "corpora.train"
|
|||
|
|
dev_corpus = "corpora.dev"
|
|||
|
|
seed = ${system.seed}
|
|||
|
|
gpu_allocator = ${system.gpu_allocator}
|
|||
|
|
frozen_components = []
|
|||
|
|
before_to_disk = null
|
|||
|
|
|
|||
|
|
[pretraining]
|
|||
|
|
|
|||
|
|
[initialize]
|
|||
|
|
vectors = ${paths.vectors}
|
|||
|
|
init_tok2vec = ${paths.init_tok2vec}
|
|||
|
|
vocab_data = null
|
|||
|
|
lookups = null
|
|||
|
|
before_init = null
|
|||
|
|
after_init = null
|
|||
|
|
|
|||
|
|
[initialize.components]
|
|||
|
|
|
|||
|
|
[initialize.components.textcat]
|
|||
|
|
labels = ['label1', 'label2']
|
|||
|
|
|
|||
|
|
[initialize.tokenizer]
|
|||
|
|
Úcomponent_namerK Útextcat_multilabeliü c ó — d„ }t « 5 }|dz } ||« t j d| « }|j d|j « « }t j
|
|||
|
|
|« }t
|
|||
|
|
|« ddd« y# 1 sw Y yxY w)z.Test intializing textcat with labels in a listc ó — t j d« }|j d« }dddœ|_ t |g¬« j « }| j
|
|||
|
|
d« 5 }|j |« d d d « y # 1 sw Y y xY w)NrD z Some textr r( )Úlabel1Úlabel2)ÚdocsÚwb)r] r^ r5 r r Úto_bytesÚopenÚwrite)Úout_filer7 rv Úout_dataÚfile_s r. Ú create_dataz#test_issue6908.<locals>.create_data se € Ü�k‰k˜$Ó ˆØ�l‰l˜;Ó'ˆØ ¨1Ñ-ˆŒÜ ˜uÔ%×.Ñ.Ó0ˆØ
|
|||
|
|
�]‰]˜4Ô
|
|||
|
|
EØ �K‰K˜Ô !÷ !×
|
|||
|
|
Ñ
|
|||
|
|
ús Á A8Á8Bz train.spacyÚTEXTCAT_PLACEHOLDERÚTRAIN_PLACEHOLDERN)r ÚCONFIG_ISSUE_6908ÚreplaceÚas_posixr Úload_config_from_strr )r� r› Útmp_pathÚ
|
|||
|
|
train_pathÚ
|
|||
|
|
config_strrM s r. Útest_issue6908r¥ ss € ò"ô
|
|||
|
|
Œ ˜8Ø
|
|||
|
|
Ñ-ˆ
|
|||
|
|
Ù�JÔ Ü&×.Ñ.Ð/DÀnÓUˆ
|
|||
|
|
Ø ×'Ñ'Ð(;¸Z×=PÑ=PÓ=RÓSˆ
|
|||
|
|
Ü×*Ñ*¨:Ó6ˆÜ�Ô÷
|
|||
|
|
|
|||
|
|
� ‰ ús ŽA$A;Á;Bik c óp — ddd dœ} t t | « ddddœd d d dœdœ} t t | d d
|
|||
|
|
¬ « y ) NgÛ0Lž™}Ù?g¼ÉÒ7Šî?)ÚLABEL_AÚLABEL_BÚLABEL_CgA£Ï¢ÒãÕ?gŠ "nN%Ù?g3ŒM 'ãß?)ÚpÚrÚf)r§ r¨ ÚfooÚbar)ÚnameÚtype)r r r
|
|||
|
|
)Úscoress r. Útest_issue7019r² sC € à#°È4Ñ
|
|||
|
|
P€FÜ œs FÔ+à"¨¸ ÑGØ D¨tÑ4ñ€Fô ”s˜F¨°UÖ;r1 i°& c óŒ — t « } | j d« }t | « }| j |« |« }|j |D �cg c] }|j
|
|||
|
|
‘Œ c}« }|j
|
|||
|
|
||« d }|j
|
|||
|
|
|dz |j dd¬« « d }|t j |« k( sJ ‚y c c}w )NrK r r )Úaxis)
|
|||
|
|
r
|
|||
|
|
r` r9 rc r‚ Ú predictedÚget_lossÚrepeatÚpytestÚapprox)r7 rK r/ rY Úegr± ÚlossÚloss_double_bss r. Útest_issue9904r½ % s° € ä
|
|||
|
|
‹*€CØ�l‰l˜9Ó%€GÜ1°#Ó6€L؇N�N�<Ô á‹~€HØ
|
|||
|
|
�_‰_±XÓ >±X¨r˜bŸl›l°XÑ >Ó
|
|||
|
|
?€Fà × Ñ ˜H fÓ -¨aÑ 0€DØ ×%Ñ% h°¡l°F·M±MÀ!È!°MÓ4LÓMÈaÑP€NØ ”6—=‘= Ó0Ò 0Ð0Ñ 0ùò ?s ÁCz#Test is flakey when run with others)Úreasonc ó6 — t « } | j d« }|j d« | j « t d« D ]# }dD ] \ }}| j |dd|iif« Œ Œ% | d« }d|j v sJ ‚|j d dk\ sJ ‚y )NrK Úanswerr} ))Úaaaar )Úbbbbr )Úaar )Ú bbbbbbbbbr )Úaaaaaar( r Úaaaç à?)r
|
|||
|
|
r` ra rc rd rf r )r7 rK rp rk rÀ rv s r. Útest_simple_trainrÈ 4 sž € ä
|
|||
|
|
‹*€CØ�l‰l˜9Ó%€GØ ×Ñ�hÔ Ø‡N�NÔÜ
|
|||
|
|
�1ŽXˆó
|
|||
|
|
‰LˆD�&ð
|
|||
|
|
�J‰J˜˜v¨°&Ð'9Ð :Ð;Õ <ñ
|
|||
|
|
ð ñ ˆe‹*€CØ �s—x‘xÑ Ð Ð Ø �8‰8�HÑ Ò $Ð$Ñ $r1 c
|
|||
|
|
óò — t j d« t j j d« g } t « }g d¢}|D ]] }|D ]V }|D �ci c] }|t ||k( « “Œ }}| j t
|
|||
|
|
|j dgdz ||gz dgdz z ¬« |f« ŒX Œ_ t j | « t |j d¬« }|D ] }|j |« Œ |j d„ « }t d « D ]W } i }
|
|||
|
|
| D � � cg c] \ } } t j | d
|
|||
|
|
i« ‘Œ }
|
|||
|
|
} } |j |
|
|||
|
|
||
|
|||
|
|
¬ « t j | « ŒY |D ]� }|D ]z }t
|
|||
|
|
|j dgdz ||gz dgdz z ¬« } |D �ci c] }|||k( “Œ
|
|||
|
|
}} || « | j j# « D ] \ } }|| s|d k rŒJ ‚|d kD rŒJ ‚ Œ| Œƒ y c c}w c c} } w c c}w )
|
|||
|
|
Nr} )ÚaÚbÚcÚdrQ )Úwordsé )Úwidthc ó — g S r* r+ r+ r1 r. ú<lambda>z0test_textcat_learns_multilabel.<locals>.<lambda>W s € ©2r1 é r ©rZ r\ rÇ )ÚrandomÚseedÚnumpyr
|
|||
|
|
Úfloatr3 r ÚvocabÚshuffler ra rc rd r r4 rf r Úitems)r“ r7 ÚlettersÚw1Úw2Úletterr rK ro rp r\ rv ÚcatrY ÚtruthÚscores r. Ú test_textcat_learns_multilabelrã H së € ä
|
|||
|
|
‡K�K�„NÜ ‡L�L×Ñ�aÔØ
|
|||
|
|
€DÜ
|
|||
|
|
‹*€CÚ €GÛˆÛˆBÙ>EÓF¹g°F�FœE "¨¡,Ó /Ñ/¸gˆDÐFØ �K‰Kœ˜SŸY™Y¨s¨e°a©i¸2¸r¸(Ñ.BÀcÀUÈQÁYÑ.NÔOÐQUÐVÕ Wñ ð ô ‡N�N�4ÔÜ ˜cŸi™i¨qÔ1€GÛˆØ×ј&Õ!ð à×"Ñ"¡:Ó.€IÜ
|
|||
|
|
�2ŽYˆØˆÙKOÔPÉ4¹x¸sÀC”G×%Ñ% c¨F°D¨>Õ:È4ˆÑPØ�‰�x Y°vˆÔ>Ü�‰�tÕ ð ó
|
|||
|
|
ˆÛˆBÜ�c—i‘i¨ u¨q¡y°B¸°8Ñ';¸s¸eÀa¹iÑ'GÔHˆCÙ8?Ó@¹¨f�V˜R 6™\Ñ)¸ˆEÐ@Ù �CŒLØ !Ÿh™hŸn™nÖ .‘
|
|||
|
|
��UØ ˜S’zØ 3›;Ð&˜;à 3›;Ð&˜;ñ /ñ ñ ùò Gùó Qùò As ÁG)Ä"G.Æ
|
|||
|
|
G4r¯ c ó‚ — t « }|j | « }|j d« t j t
|
|||
|
|
« 5 |j d« d d d « | dk( r3t j t
|
|||
|
|
« 5 |j
|
|||
|
|
« d d d « y |j
|
|||
|
|
« y # 1 sw Y ŒRxY w# 1 sw Y y xY w)NrÀ é rK )r
|
|||
|
|
r` ra r¸ ÚraisesÚ
|
|||
|
|
ValueErrorrc )r¯ r7 rK s r. Útest_label_typesrè i sŠ € ä
|
|||
|
|
‹*€CØ�l‰l˜4Ó €GØ ×Ñ�hÔ Ü � ‰ ”zÕ "Ø×ј!Ô ÷
|
|||
|
|
#ð ˆyÒÜ
|
|||
|
|
�]‰]œ:Õ
|
|||
|
|
&Ø �N‰NÔ ÷ 'Ð
|
|||
|
|
&ð �‰Õ÷
|
|||
|
|
#Ð "ú÷ 'Ð
|
|||
|
|
&ús ÁB)Á>B5Â)B2Â5B>zname,get_examplesc óÖ ‡— t « }|j | « } ||« Šˆfd„}t j t « 5 |j |¬« d d d « y # 1 sw Y y xY w)Nc óž •— ‰« } | d j }t |j j « « d }d|j |< | S )Nr g @)Ú referenceÚlistr Úkeys)rY ÚrefÚkeyÚexample_getters €r. Úinvalid_examplesz2test_invalid_label_value.<locals>.invalid_examples„ sE ø€ á!Ó#ˆØ�q‰k×#Ñ#ˆÜ�3—8‘8—=‘=“?Ó# AÑ&ˆØˆ�‰�‰
|
|||
|
|
؈r1 ©r/ ©r
|
|||
|
|
r` r¸ ræ rç rc )r¯ r/ r7 rK rñ rð s @r. Útest_invalid_label_valuerô x sR ø€ ô ‹*€CØ�l‰l˜4Ó €GÙ! #Ó&€Nôô
|
|||
|
|
� ‰ ”zÕ "Ø �‰Ð$4ˆÔ5÷
|
|||
|
|
#× "Ñ "ús ÁA Á A(c ó¶ — t « }|j | « t j t « 5 |j « d d d « y # 1 sw Y y xY wr* ró )r¯ r7 s r. Ú
|
|||
|
|
test_no_labelrö � s7 € ä
|
|||
|
|
‹*€C؇L�L�ÔÜ � ‰ ”zÕ "Ø �‰Ô÷
|
|||
|
|
#× "Ñ "ús µAÁAc ój — t « }|j | « |j ||« ¬« y )Nrò )r
|
|||
|
|
r` rc )r¯ r/ r7 s r. Útest_implicit_labelrø ˜ s* € ô ‹*€C؇L�L�Ô؇N�N¡ ¨SÓ 1€NÕ2r1 zname,textcat_configrE TFrQ )rG rH rJ rI zspacy.TextCatEnsemble.v1é@ iÐ r( ) rG rH Úpretrained_vectorsrÐ Ú
|
|||
|
|
embed_sizeÚ
|
|||
|
|
conv_depthÚ window_sizerI Údropoutzspacy.TextCatEnsemble.v2)rG Útok2vecÚ linear_modelzspacy.TextCatCNN.v1)rG rÿ rH c óf — t « }d|i}|j | |¬« }|j d« |j d« |j « |j j d« dv sJ ‚t
|
|||
|
|
j t « 5 |j d« ddd« y# 1 sw Y yxY w) z/The old textcat architectures weren't resizabler& ©rM r! r" ÚnO©r NÚNEUTRALN) r
|
|||
|
|
r` ra rc r& Ú
|
|||
|
|
maybe_get_dimr¸ ræ rç ©r¯ rx r7 Ú pipe_configrK s r. Útest_no_resizer ¦ s� € ô2 ‹*€CؘNÐ+€KØ�l‰l˜4¨ ˆlÓ4€GØ ×Ñ�jÔ!Ø ×Ñ�jÔ!؇N�NÔØ �=‰=× &Ñ & tÓ ,° Ñ 9Ð9Ð 9ä � ‰ ”zÕ "Ø×ј)Ô$÷
|
|||
|
|
#× "Ñ "ús  B'Â'B0zspacy.TextCatBOW.v3zspacy.TextCatReduce.v1)rG rÿ rH Úuse_reduce_firstÚuse_reduce_lastÚuse_reduce_maxÚuse_reduce_meanc ó† — t « }d|i}|j | |¬« }|j d« |j d« |j j d« dv sJ ‚|j « |j j d« dv sJ ‚|j d« |j j d« dv sJ ‚y )
|
|||
|
|
z+The new textcat architectures are resizabler& r r! r" r r r )rQ NN)r
|
|||
|
|
r` ra r& r rc r s r. Ú test_resizer Ì s² € ô ‹*€CؘNÐ+€KØ�l‰l˜4¨ ˆlÓ4€GØ ×Ñ�jÔ!Ø ×Ñ�jÔ!Ø �=‰=× &Ñ & tÓ ,° Ñ 9Ð9Ð 9؇N�NÔØ �=‰=× &Ñ & tÓ ,° Ñ 9Ð9Ð 9Ø ×Ñ�iÔ Ø �=‰=× &Ñ & tÓ ,° Ñ 9Ð9Ñ 9r1 c ó& ‡— t d« t « }d|i}|j | |¬« }g Št D ]9 \ }}‰j t j |j |« |« « Œ; |j ˆfd„¬« }|j j d« dv sJ ‚t d« D ] }i } |j ‰|| ¬ « Œ d
|
|||
|
|
}
|
|||
|
|
||
|
|||
|
|
« } t | j « d k( sJ ‚| j d } | j d
|
|||
|
|
}
|
|||
|
|
|j d« ||
|
|||
|
|
« } t | j « dk( sJ ‚| j d | k( sJ ‚| j d
|
|||
|
|
|
|
|||
|
|
k( sJ ‚| j d dk sJ ‚t d« D ] }i } |j ‰|| ¬ « Œ ||
|
|||
|
|
« } t | j « dk( sJ ‚| j d | k7 sJ ‚| j d
|
|||
|
|
|
|
|||
|
|
k7 sJ ‚| j D ] }| j | dk rŒJ ‚ y )Nr r& r c ó •— ‰ S r* r+ r, s €r. rÒ z*test_resize_same_results.<locals>.<lambda> ó ø€ ±Nr1 rò r r r} rÔ ú I am happy.r r! r" r rQ r( )r r r` r2 r3 r r4 r5 rc r& r rd rf rƒ r ra )r¯ rx r7 r rK rk Ú annotationsro rp r\ Ú test_textrv Úpos_predÚneg_predrà r- s @r. Útest_resize_same_resultsr é s ø€ ô �AÔÜ
|
|||
|
|
‹)€CؘNÐ+€KØ�l‰l˜4¨ ˆlÓ4€Gà€Nß 4шˆkØ× Ñ œg× /Ñ /°· ± ¸TÓ0BÀKÓ PÕQð 5à—‘Ó,B�ÓC€IØ �=‰=× &Ñ & tÓ ,° Ñ 9Ð9Ð 9ä
|
|||
|
|
�1ŽXˆØˆØ �
|
|||
|
|
‰
|
|||
|
|
�> y¸ˆ
|
|||
|
|
Õ@ð ð
|
|||
|
|
€IÙ
|
|||
|
|
|
|||
|
|
ˆi‹.€CÜ ˆs�x‰x‹=˜AÒ Ð Ð Ø�x‰x˜
|
|||
|
|
Ñ#€HØ�x‰x˜
|
|||
|
|
Ñ#€Hð ×Ñ�iÔ Ù
|
|||
|
|
|
|||
|
|
ˆi‹.€CÜ ˆs�x‰x‹=˜AÒ Ð Ð Ø �8‰8�JÑ 8Ò +Ð+Ð +Ø �8‰8�JÑ 8Ò +Ð+Ð +Ø �8‰8�IÑ !Ò #Ð#Ð #ä
|
|||
|
|
�1ŽXˆØˆØ �
|
|||
|
|
‰
|
|||
|
|
�> y¸ˆ
|
|||
|
|
Õ@ð ñ
|
|||
|
|
ˆi‹.€CÜ ˆs�x‰x‹=˜AÒ Ð Ð Ø �8‰8�JÑ 8Ò +Ð+Ð +Ø �8‰8�JÑ 8Ò +Ð+Ð +Ø�xŒxˆØ�x‰x˜‰} Ó!Ð!Ð!ñ r1 c óJ ‡— t « } | j d« g Št D ]9 \ }}‰j t j
|
|||
|
|
| j
|
|||
|
|
|« |« « Œ; t j t « 5 | j ˆfd„¬« d d d « y # 1 sw Y y xY w)NrK c ó •— ‰ S r* r+ r, s €r. rÒ z.test_error_with_multi_labels.<locals>.<lambda>+ s ø€ ©Nr1 rò ) r
|
|||
|
|
r` r< r3 r r4 r5 r¸ ræ rç rc )r7 rk r r- s @r. Ú test_error_with_multi_labelsr $ su ø€ Ü
|
|||
|
|
‹*€C؇L�L�ÔØ€Nß 3шˆkØ× Ñ œg× /Ñ /°· ± ¸TÓ0BÀKÓ PÕQð 4ä � ‰ ”zÕ "Ø �‰Ó$:ˆÔ;÷
|
|||
|
|
#× "Ñ "ús Á:BÂB"rF r} z#spacy.TextCatParametricAttention.v1c óÎ — t « }d|i}|j | |¬« }|j d« |j d« |j « |j dg« y )Nr& r r! r" zThis is a test.)r r` ra rc r‰ r s r. Útest_tok2vec_lazy_initr / s] € ô$ ‹)€CؘNÐ+€KØ�l‰l˜4¨ ˆlÓ4€GØ ×Ñ�jÔ!Ø ×Ñ�jÔ!؇N�NÔ؇H�HÐ Ð
|
|||
|
|
Õ!r1 z name,get_examples, train_datac ó — t « }|j | « }|D ]= \ }}|j d« j « D ] \ }}|j |« Œ Œ? |j « |j ||« ¬« t
|
|||
|
|
j t « 5 |j d„ ¬« d d d « t
|
|||
|
|
j t « 5 |j |« ¬« d d d « y # 1 sw Y ŒCxY w# 1 sw Y y xY w)Nr rò c ó — y r* r+ r+ r1 r. rÒ z*test_initialize_examples.<locals>.<lambda>[ s € ¨Dr1 ) r
|
|||
|
|
r` ÚgetrÛ ra rc r¸ ræ Ú TypeError) r¯ r/ rj r7 rK rk r rm Úvalues r. Útest_initialize_examplesr# J sÈ € ô ‹*€CØ�l‰l˜4Ó €GÛ 'шˆkØ 'ŸO™O¨FÓ 3× 9Ñ 9Ö ;‰LˆE�5Ø × Ñ ˜eÕ $ñ <ð (ð ‡N�NÔ؇N�N¡ ¨SÓ 1€NÔ2Ü � ‰ ”yÕ !Ø �‰¡LˆÔ1÷
|
|||
|
|
"ä � ‰ ”yÕ !Ø �‰¡L£NˆÔ3÷
|
|||
|
|
"Ð !÷
|
|||
|
|
"Ð !úç !Ð !ús  C5ÃDÃ5C>ÄD
|
|||
|
|
c ó¢ ‡— t d« t « } | j d« }g Št D ]9 \ }}‰j t j | j |« |« « Œ; | j ˆfd„¬« }|j j d« dk( sJ ‚t d« D ] }i }| j ‰||¬« Œ d d k sJ ‚d
|
|||
|
|
} | |« }|j } | d d kD sJ ‚| d | d
|
|||
|
|
z t j dd« k( sJ ‚t! « 5 }
|
|||
|
|
| j# |
|
|||
|
|
« t% j& |
|
|||
|
|
« } | |« } | j }
|
|||
|
|
|
|
|||
|
|
d d kD sJ ‚|
|
|||
|
|
d |
|
|||
|
|
d
|
|||
|
|
z t j dd« k( sJ ‚ d d d « | j) ‰« }|d dk( sJ ‚|d dk( sJ ‚|d dk( sJ ‚|d dk( sJ ‚d|v sJ ‚g d¢}| j+ |« D �cg c] }|j ‘Œ }}| j+ |« D �cg c] }|j ‘Œ }}|D �cg c]
|
|||
|
|
} | |« ‘Œ c}D �cg c] }|j ‘Œ }}t- ||« D ] \ }}|D ] }t/ || || d¬« Œ Œ" t- ||« D ] \ }}|D ] }t/ || || d¬« Œ Œ" y # 1 sw Y �Œ)xY wc c}w c c}w c c}w c c}w )Nr rK c ó •— ‰ S r* r+ r, s €r. rÒ z%test_overfitting_IO.<locals>.<lambda>i r r1 rò r r é2 rÔ ç{®Gáz„?r r! çÍÌÌÌÌÌì?r" r gü©ñÒMbP?Ú cats_micro_fÚ cats_macro_fÚcats_macro_aucÚ
|
|||
|
|
cats_scoreÚcats_score_desc©zJust a sentence.zI like green eggs.r z
|
|||
|
|
I eat ham.r} r~ )r r r` r2 r3 r r4 r5 rc r& Úget_dimrd rf r r¸ r¹ r Úto_diskr Úload_model_from_pathÚevaluater‰ r_ r )r7 rK rk r ro rp r\ r rv r Útmp_dirÚnlp2Údoc2Úcats2r± ÚtextsÚ batch_cats_1Ú batch_cats_2Ú
|