66 lines
14 KiB
Plaintext
66 lines
14 KiB
Plaintext
|
Ë
|
|||
|
|
>û gG% ã ó† — d dl mZ d dlmZmZmZmZmZmZm Z m
|
|||
|
|
Z
|
|||
|
|
d dl Z d dl m
|
|||
|
|
Z
|
|||
|
|
mZmZmZmZmZmZmZmZmZmZmZ d dlmZ d dlm Z m Z ddl m Z m Z dd l!m"Z" dd
|
|||
|
|
l#m$Z$m%Z% dd l&m'Z( er dd l)m*Z* dd
|
|||
|
|
l+m,Z, e%jZ d« de.de.de/dedegef fd„« Z0 e%jZ d« de.de.de.dedegef fd„« Z1d„ Z2d„ Z3 d.dede.de.d ee. def
|
|||
|
|
d „Z4d ddede.de.def
|
|||
|
|
d „Z5d ddede.de.d e.def d!„Z6 d/d dd"ed#e7defd$„Z8 G d%„ d&« Z9 d/d'ed( d)e9d#e7de e jt ed( f fd*„Z;d0d+e/d)e9d,e/de/fd-„Z<y)1é )Úpartial)Ú
|
|||
|
|
TYPE_CHECKINGÚAnyÚCallableÚIterableÚListÚOptionalÚTupleÚcastN) ÚCosineDistanceÚ
|
|||
|
|
L2DistanceÚ LayerNormÚLinearÚMaxoutÚModelÚ MultiSoftmaxÚSoftmaxÚchainÚ
|
|||
|
|
list2arrayÚto_categoricalÚ zero_init)ÚLoss)ÚFloats2dÚInts1dé )ÚIDÚORTH)ÚErrors)ÚOOV_RANKÚregistry)ÚMode©ÚDoc)ÚVocabzspacy.PretrainVectors.v1Ú
|
|||
|
|
maxout_piecesÚ hidden_sizeÚlossÚreturnr$ c óR ‡ ‡‡‡— dddt dt fˆˆˆ fd„ }dt fˆfd„ Š|S )NÚvocabr$ Útok2vecr( c ó¸ •— | j j d dk( rt t j « ‚t | |‰‰¬« } ‰« |j d< |S )Né r )r& r% r' )ÚvectorsÚshapeÚ
|
|||
|
|
ValueErrorr ÚE875Ú build_cloze_multi_task_modelÚattrs)r* r+ ÚmodelÚcreate_vectors_lossr& r% s €€€ú[C:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/ml/models/multi_task.pyÚcreate_vectors_objectivez9create_pretrain_vectors.<locals>.create_vectors_objective% sT ø€ Ø �=‰=× Ñ ˜qÑ ! QÒ &Ü œVŸ[™[Ó)Ð )Ü,Ø �7¨ À=ô
|
|||
|
|
ˆñ 2Ó 3ˆ� ‰ �FÑ؈ ó c óâ •— ‰dk( r t dd¬« } t t | ¬« S ‰dk( r t d¬« } t t | ¬« S t t
|
|||
|
|
j j ‰d¬« « ‚) NÚcosineT)Ú normalizeÚ ignore_zeros)ÚdistanceÚL2)r; z'cosine', 'L2')ÚfoundÚ supported)r r Úget_vectors_lossr
|
|||
|
|
r0 r ÚE906Úformat)r= r' s €r6 r5 z4create_pretrain_vectors.<locals>.create_vectors_loss. sd ø€ à �8Ò Ü%°À4ÔHˆHÜÔ+°hÔ?Ð ?Ø
|
|||
|
|
�TŠ\Ü!¨DÔ1ˆHÜÔ+°hÔ?Ð ?ä œVŸ[™[× /Ñ /°dÐFVÐ /Ó WÓXÐ Xr8 )r r )r% r& r' r7 r5 s ``` @r6 Úcreate_pretrain_vectorsrD ! s3 û€ ð¨ð ¼%ð ÄE÷ ð Y¤õ Yð $Ð#r8 zspacy.PretrainCharacters.v1Ú n_charactersc ó8 ‡ ‡‡— dddt dt fˆˆ ˆfd„ }|S )Nr* r$ r+ r( c ód •— t | |‰‰‰¬« }t t ‰¬« |j d< |S )N)r& r% Únr_char©rH r' )Ú'build_cloze_characters_multi_task_modelr Úget_characters_lossr3 )r* r+ r4 r& r% rE s €€€r6 Úcreate_characters_objectivez?create_pretrain_characters.<locals>.create_characters_objective@ s9 ø€ Ü7Ø Ø Ø#Ø'Ø ô
|
|||
|
|
ˆô &Ô&9À<Ô Pˆ� ‰ �FÑ؈ r8 )r )r% r& rE rL s ``` r6 Úcreate_pretrain_charactersrM < s% ú€ ð ¨7ð ¼Uð Äu÷ ð 'Ð&r8 c óF — |d j }|j j t j k( r‚| j |D �cg c]% }|j
|
|||
|
|
t « j « ‘Œ' c}« }|d j j j | }d||t k( < |||« \ }} | |fS |j j t j k( r|| j |D �cg c]% }t t |j
|
|||
|
|
t « « ‘Œ' c}« }
|
|||
|
|
|j j |
|
|||
|
|
« }| j! |« } |||« \ }} | |fS t# t$ j& j) |j j ¬« « ‚c c}w c c}w )z^Compute a loss based on a distance between the documents' vectors and
|
|||
|
|
the prediction.
|
|||
|
|
r )Úmode)r* r. rO Ú VectorsModeÚdefaultÚflattenÚto_arrayr ÚravelÚdatar Úfloretr r r Ú get_batchÚ as_contigr0 r ÚE850rC ) ÚopsÚdocsÚ
|
|||
|
|
predictionr= r* ÚdocÚidsÚtargetÚd_targetr' Úkeyss r6 rA rA N s\ € ð
|
|||
|
|
�‰G�M‰M€EØ ‡}�}×Ñœ[× 0Ñ 0Ò0ð
|
|||
|
|
�k‰k¹tÓD¹t¸˜3Ÿ<™<¬Ó+×1Ñ1Õ3¸tÑDÓEˆØ�a‘— ‘ ×&Ñ&×+Ñ+¨CÑ0ˆØ"#ˆˆs”h‰Ñ Ù! *¨fÓ5‰ˆ�$ð �ˆ>Ðð
|
|||
|
|
� ‰ × Ñ œ{× 1Ñ 1Ò 1Ø�{‰{ÉÓMÉÀœD¤¨¯ © ´dÓ);Õ <ÈÑMÓNˆØ— ‘ ×(Ñ(¨Ó.ˆØ— ‘ ˜vÓ&ˆÙ! *¨fÓ5‰ˆ�$ð �ˆ>Ðô œŸ™×+Ñ+°· ± ×1CÑ1CÐ+ÓDÓEÐEùò Eùò
|
|||
|
|
Ns Á*FÃ/*F c ój — t j |D �cg c] }|j |¬« ‘Œ c}« }|j d« }| j t |d¬« d¬« }|j dd|z f« }||z
|
|||
|
|
}|dz j
|
|||
|
|
« }|t |j d « z } || fS c c}w )
|
|||
|
|
zGCompute a loss based on a number of characters predicted from the docs.rI )éÿÿÿÿé )Ú n_classesÚf©Údtyperc é r ) ÚnumpyÚvstackÚ
|
|||
|
|
to_utf8_arrayÚreshapeÚasarrayr ÚsumÚfloatr/ )
|
|||
|
|
rZ r[ r\ rH r] Ú
|
|||
|
|
target_idsr_ Údiffr' r` s
|
|||
|
|
r6 rK rK f s° € ä— ‘ ÉTÓ RÉTÀc˜s× 0Ñ 0¸Ð 0Õ AÈTÑ RÓS€JØ×#Ñ# EÓ*€JØ
|
|||
|
|
�[‰[œ ¨
|
|||
|
|
¸cÔBÈ#ˆ[Ó
|
|||
|
|
N€FØ
|
|||
|
|
�^‰^˜R w¡ Ð /Ó
|
|||
|
|
0€FØ ˜Ñ €DØ �!‰G�=‰=‹?€DØ”e˜J× ,Ñ ,¨QÑ /Ó0Ñ0€HØ �ˆ>Ðùò Ss ”B0r+ Útoken_vector_widthÚnOc ó — t ||dz ¬« }t | t |dz ||d¬« t |dz « |« }|j d| « |j d|« |S )Nri )rt ÚnIç )rt rv ÚnPÚdropoutr+ Ú output_layer)r r r r Úset_ref)r+ r% rs rt Úsoftmaxr4 s r6 Úbuild_multi_task_modelr} r sr € ô ˜Ð 2°QÑ 6Ô7€GÜ ØÜØ! AÑ%Ø!Ø Øô
|
|||
|
|
ô Ð$ qÑ(Ó)Øó
|
|||
|
|
|
|||
|
|
€Eð
|
|||
|
|
‡M�M�)˜WÔ%Ø ‡M�M�. 'Ô*Ø €Lr8 r* c
|
|||
|
|
óp — | j j d }t t t t
|
|||
|
|
d t f t « « t ||j d« |dd¬« t ||t ¬« « }t ||« }t | |« }|j d|« |j d |« |S )
|
|||
|
|
Nr- r rt Trw )rt rv rx r; ry )rt rv Úinit_Wr+ rz )r. r/ r r r r r r r Úget_dimr r Úbuild_masked_language_modelr{ )r* r+ r% r& rt rz r4 s r6 r2 r2 ‰ s¦ € ð
|
|||
|
|
� ‰ × Ñ ˜QÑ €BÜÜ ŒU”4˜
|
|||
|
|
Ñ#¤XÐ-Ñ
|
|||
|
|
.´
|
|||
|
|
³ Ó=ÜØØ�‰˜tÓ$Ø ØØô
|
|||
|
|
ô �"˜¬YÔ7ó
|
|||
|
|
€Lô
|
|||
|
|
�'˜<Ó (€EÜ '¨¨uÓ 5€EØ ‡M�M�)˜WÔ%Ø ‡M�M�. ,Ô/Ø €Lr8 rH c ó* — t t t t d t f t « « t
|
|||
|
|
||¬« t |¬« t dg|z |¬« « }t | t ||« « }|j d|« |j d|« |S )Nr )rt rx )rv rd r+ rz ) r r r r r r r r r r� r{ )r* r+ r% r& rH rz r4 s r6 rJ rJ Ÿ s� € ô Ü ŒU”4˜
|
|||
|
|
Ñ#¤XÐ-Ñ
|
|||
|
|
.´
|
|||
|
|
³ Ó=Ü�+ -Ô0Ü�[Ô!Ü�c�U˜W‘_¨Ô5ó €Lô
|
|||
|
|
(¨¬u°W¸lÓ/KÓ L€EØ ‡M�M�)˜WÔ%Ø ‡M�M�. ,Ô/Ø €Lr8 Ú
|
|||
|
|
wrapped_modelÚ mask_probc óÊ ‡‡— t | « Šˆˆfd„}ddt fd„}t d||g|d|i|j D �ci c] }|d“Œ c}¬« }|j d|« |S c c}w ) z7Convert a model into a BERT-style masked language modelc óâ •‡‡— t |‰‰¬« \ Š}| j j ‰« j ‰j d df« Š | j
|
|||
|
|
d ||« \ }Šˆˆfd„}||fS )N)r„ r r- c ó$ •— | d‰z
|
|||
|
|
z } ‰| « S )Nr- © )Úd_outputÚbackpropÚmasks €€r6 Ú mlm_backwardzFbuild_masked_language_model.<locals>.mlm_forward.<locals>.mlm_backward¹ s ø€ Ø ˜˜D™Ñ ˆHÙ˜HÓ%Ð %r8 )Ú _apply_maskrZ rn rm r/ Úlayers) r4 r[ Úis_trainÚoutputrŒ rŠ r‹ r„ Ú random_wordss @@€€r6 Ú mlm_forwardz0build_masked_language_model.<locals>.mlm_forward´ sn ú€ Ü |¸yÔI‰
|
|||
|
|
ˆˆdØ�y‰y× Ñ Ó&×.Ñ.°·
|