174 lines
24 KiB
Plaintext
174 lines
24 KiB
Plaintext
|
Ë
|
|||
|
|
>û g
|
|||
|
|
K ã ó@ — d dl Z d dlZd dlmZ d dlmZ d dlmZmZm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z m
|
|||
|
|
Z
|
|||
|
|
d dlZd dlmZmZmZmZ d dlmZ ddlmZ dd lmZmZ dd
|
|||
|
|
lm Z m Z dd l m Z dd l m!Z! dd
|
|||
|
|
l"m#Z# ddl$m%Z%m&Z& ddl'm(Z(m)Z)m*Z* ddlm+Z+ ddl,m-Z- ddl.m/Z/ ddl0m1Z1 dZ2d„ Z3 G d„ de1« Z4y)é N)Úislice)ÚPath)ÚAnyÚCallableÚDictÚIterableÚListÚOptionalÚUnion)ÚCosineDistanceÚModelÚ OptimizerÚset_dropout_rate)ÚFloats2dé )Úutil)ÚErrorsÚWarnings)Ú CandidateÚ
|
|||
|
|
KnowledgeBase)ÚLanguage)Úempty_kb)ÚScorer)ÚDocÚSpan)ÚExampleÚvalidate_examplesÚvalidate_get_examples)ÚSimpleFrozenList)ÚVocabé )Údeserialize_config)Ú
|
|||
|
|
TrainablePipeTc óP — t j | fdt j gi|¤ŽS )NÚnegative_labels)r Ú score_linksÚEntityLinker_v1ÚNIL)ÚexamplesÚkwargss údC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/pipeline/legacy/entity_linker.pyÚentity_linker_scorer, s' € Ü × Ñ ˜hÑ X¼ ×9LÑ9LÐ8MÐ XÐQWÑ XÐXó c ó" — e Zd ZdZdZ d0eedœdedede de
|
|||
|
|
e de d e d
|
|||
|
|
e d e d e
|
|||
|
|
eege
|
|||
|
|
e f d
|
|||
|
|
e dee
|
|||
|
|
ddfd„Zde
|
|||
|
|
egef fd„Zd1d„Zdddœde
|
|||
|
|
g e
|
|||
|
|
e f dee dee
|
|||
|
|
egef fd„Zddddœde
|
|||
|
|
e d ed ee d eee ef dee ef f
|
|||
|
|
d „Zde
|
|||
|
|
e d e fd!„Z d"e
|
|||
|
|
e de e fd#„Z d"e
|
|||
|
|
e d$e e ddfd%„Z! e"« d&œd'„Z# e"« d&œd(„Z$ e%« d&œd)e&e e'f d*e
|
|||
|
|
e ddfd+„Z( e%« d&œd)e&e e'f d*e
|
|||
|
|
e dd fd,„Z)ddd-œd.„Z*d/„ Z+y)2r' z^Pipeline component for named entity linking.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker
|
|||
|
|
r( )Ú overwriteÚscorerÚvocabÚmodelÚnameÚlabels_discardÚn_sentsÚ
|
|||
|
|
incl_priorÚ incl_contextÚentity_vector_lengthÚget_candidatesr/ r0 ÚreturnNc ó — || _ || _ || _ t |« | _ || _ || _ || _ | | _ d|
|
|||
|
|
i| _ t d¬« | _ t |« | j « | _
|
|||
|
|
| | _ y)a Initialize an entity linker.
|
|||
|
|
|
|||
|
|
vocab (Vocab): The shared vocabulary.
|
|||
|
|
model (thinc.api.Model): The Thinc Model powering the pipeline component.
|
|||
|
|
name (str): The component instance name, used to add entries to the
|
|||
|
|
losses during training.
|
|||
|
|
labels_discard (Iterable[str]): NER labels that will automatically get a "NIL" prediction.
|
|||
|
|
n_sents (int): The number of neighbouring sentences to take into account.
|
|||
|
|
incl_prior (bool): Whether or not to include prior probabilities from the KB in the model.
|
|||
|
|
incl_context (bool): Whether or not to include the local context in the model.
|
|||
|
|
entity_vector_length (int): Size of encoding vectors in the KB.
|
|||
|
|
get_candidates (Callable[[KnowledgeBase, Span], Iterable[Candidate]]): Function that
|
|||
|
|
produces a list of candidates, given a certain knowledge base and a textual mention.
|
|||
|
|
scorer (Optional[Callable]): The scoring method. Defaults to Scorer.score_links.
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#init
|
|||
|
|
r/ F)Ú normalizeN)r1 r2 r3 Úlistr4 r5 r6 r7 r9 Úcfgr Údistancer Úkbr0 ) Úselfr1 r2 r3 r4 r5 r6 r7 r8 r9 r/ r0 s r+ Ú__init__zEntityLinker_v1.__init__+ s} € ð> ˆŒ
|
|||
|
|
؈Œ
|
|||
|
|
؈Œ Ü " >Ó 2ˆÔØ ˆŒ Ø$ˆŒØ (ˆÔØ ,ˆÔØ$/°Ð#;ˆŒÜ&°Ô7ˆŒ
|
|||
|
|
ð 1”(Ð/Ó0°·±Ó<ˆŒØ ˆ� r- Ú kb_loaderc ó¬ — t |« s2t t j j t |« ¬« « ‚ || j « | _ y)ziDefine the KB of this pipe by providing a function that will
|
|||
|
|
create it using this object's vocab.)Úarg_typeN)ÚcallableÚ
|
|||
|
|
ValueErrorr ÚE885ÚformatÚtyper1 r@ )rA rC s r+ Úset_kbzEntityLinker_v1.set_kbY s= € ô ˜ Ô"Ü œVŸ[™[× /Ñ /¼¸i» Ð /Ó IÓJÐ Já˜DŸJ™JÓ'ˆ�r- c ó — | j €3t t j j | j
|
|||
|
|
¬« « ‚t
|
|||
|
|
| j « dk( r3t t j j | j
|
|||
|
|
¬« « ‚y )N©r3 r )r@ rG r ÚE1018rI r3 ÚlenÚE139©rA s r+ Ú validate_kbzEntityLinker_v1.validate_kba sb € à �7‰7ˆ?Ü œVŸ\™\× 0Ñ 0°d·i±iÐ 0Ó @ÓAÐ AÜ ˆt�w‰w‹<˜1Ò Ü œVŸ[™[× /Ñ /°T·Y±YÐ /Ó ?Ó@Ð @ð r- )ÚnlprC Ú get_examplesrS c óØ — t |d« |�| j |« | j « | j j }g }g }t |« d« D ]Q }|j
|
|||
|
|
|j « |j
|
|||
|
|
| j j j |« « ŒS t |« dkD s/J t j j | j ¬« « ‚t |« dkD s/J t j j | j ¬« « ‚| j j! || j j j# |d¬« ¬« y) a— Initialize the pipe for training, using a representative set
|
|||
|
|
of data examples.
|
|||
|
|
|
|||
|
|
get_examples (Callable[[], Iterable[Example]]): Function that
|
|||
|
|
returns a representative sample of gold-standard Example objects.
|
|||
|
|
nlp (Language): The current nlp object the component is part of.
|
|||
|
|
kb_loader (Callable[[Vocab], KnowledgeBase]): A function that creates an InMemoryLookupKB from a Vocab instance.
|
|||
|
|
Note that providing this argument, will overwrite all data accumulated in the current KB.
|
|||
|
|
Use this only when loading a KB as-such from file.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#initialize
|
|||
|
|
zEntityLinker_v1.initializeNé
|
|||
|
|
r rM Úfloat32)Údtype)ÚXÚY)r rK rR r@ r8 r ÚappendÚxr2 ÚopsÚalloc1frO r ÚE923rI r3 Ú
|
|||
|
|
initializeÚasarray)rA rT rS rC ÚnOÚ
|
|||
|
|
doc_sampleÚ
|
|||
|
|
vector_sampleÚexamples r+ r` zEntityLinker_v1.initializeh s € ô& ˜lÐ,HÔIØ Ð Ø �K‰K˜ Ô "Ø ×ÑÔØ
|
|||
|
|
�W‰W×
|
|||
|
|
)Ñ
|
|||
|
|
)ˆØˆ
|
|||
|
|
؈
|
|||
|
|
Ü ™l›n¨bÖ1ˆGØ × Ñ ˜gŸi™iÔ (Ø × Ñ §¡§ ¡ ×!7Ñ!7¸Ó!;Õ <ð 2ô �:‹ Ò"ÐF¤F§K¡K×$6Ñ$6¸D¿I¹IÐ$6Ó$FÓFÐ"Ü�=Ó! AÒ%ÐI¤v§{¡{×'9Ñ'9¸t¿y¹yÐ'9Ó'IÓIÐ%Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ Ø˜DŸJ™JŸN™N× 2Ñ 2°=È Ð 2Ó Rð õ
|
|||
|
|
r- ç )ÚdropÚsgdÚlossesr) rg rh ri c ól — | j « |€i }|j | j d« |s|S t |d« g }|D �] }|j j
|
|||
|
|
D �cg c] }|‘Œ }}|j
|
|||
|
|
dd¬« } |j j D ]º }
|
|||
|
|
| |
|
|||
|
|
j } | sΠ|j |
|
|||
|
|
j « } t d| | j z
|
|||
|
|
« }
|
|||
|
|
t# t% |« dz
|
|||
|
|
| | j z « }||
|
|||
|
|
j }|| j& }|j( || j+ « }|j- |« Œ¼ �Œ t/ | j0 |« |s5t3 j4 t6 j8 j; d ¬
|
|||
|
|
« « |S | j0 j= |« \ }}| j? ||¬ « \ }} ||« |�| jA |« || j xx |z
|
|||
|
|
cc< |S c c}w # t $ r t t j « d‚w xY w) a. Learn from a batch of documents and gold-standard information,
|
|||
|
|
updating the pipe's model. Delegates to predict and get_loss.
|
|||
|
|
|
|||
|
|
examples (Iterable[Example]): A batch of Example objects.
|
|||
|
|
drop (float): The dropout rate.
|
|||
|
|
sgd (thinc.api.Optimizer): The optimizer.
|
|||
|
|
losses (Dict[str, float]): Optional record of the loss during training.
|
|||
|
|
Updated using the component name as the key.
|
|||
|
|
RETURNS (Dict[str, float]): The updated losses dictionary.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#update
|
|||
|
|
Nrf zEntityLinker_v1.updateÚ ENT_KB_IDT©Ú as_stringr é z
|
|||
|
|
Entity LinkerrM )Úsentence_encodingsr) )!rR Ú
|
|||
|
|
setdefaultr3 r Ú referenceÚsentsÚ get_alignedÚentsÚstartÚindexÚsentÚAttributeErrorÚ RuntimeErrorr ÚE030Úmaxr5 ÚminrO ÚendÚ predictedÚas_docr[ r r2 ÚwarningsÚwarnr ÚW093rI Ú begin_updateÚget_lossÚ
|
|||
|
|
finish_update)rA r) rg rh ri Ú
|
|||
|
|
sentence_docsÚegÚsÚ sentencesÚkb_idsÚentÚkb_idÚ
|
|||
|
|
sent_indexÚstart_sentenceÚ end_sentenceÚ start_tokenÚ end_tokenÚsent_docro Ú
|
|||
|
|
bp_contextÚlossÚd_scoress r+ ÚupdatezEntityLinker_v1.update‹ s € ð(
|
|||
|
|
×ÑÔØ ˆ>؈FØ×ј$Ÿ)™) SÔ)Ù؈Mܘ(Ð$<Ô=؈
|
|||
|
|
܈BØ$&§L¡L×$6Ò$6Ó7Ñ$6˜qšÐ$6ˆIÐ7Ø—^‘^ K¸4�^Ó@ˆFØ—|‘|×(Ô(�à ˜sŸy™yÑ)�ÚðBà%.§_¡_°S·X±XÓ%>˜
|
|||
|
|
ô
|
|||
|
|
&)¨¨J¸¿ ¹ Ñ,EÓ%F�Nä#&¤s¨9£~¸Ñ'9¸:ÈÏ É Ñ;TÓ#U�Là"+¨NÑ";×"AÑ"A�KØ )¨,Ñ 7× ;Ñ ;�Ià !Ÿ|™|¨K¸ Ð B× IÑ IÓ K�HØ!×(Ñ(¨Õ2ò' )ð ô. ˜Ÿ™ TÔ*Ù Ü �M‰Mœ(Ÿ-™-×.Ñ.°OÐ.ÓDÔ E؈MØ)-¯©×)@Ñ)@À Ó)OÑ&ИJØ Ÿ ™ Ø 1¸Hð 'ó
|
|||
|
|
‰ˆˆhñ �8Ô Ø ˆ?Ø × Ñ ˜sÔ #؈t�y‰yÓ˜TÑ!Ó؈
|
|||
|
|
ùòE 8øô *ò Bä *¬6¯;©;Ó 7¸TÐAðBús Á! H
|
|||
|
|
Â+HÈ$H3ro c ó~ — t |d« g }|D ]n }|j dd¬« }|j j D ]@ }||j }|sŒ| j
|
|||
|
|
j
|
|||
|
|
|« }|j |« ŒB Œp | j j j |« }|j |j k7 r,t j j dd¬« } t | « ‚| j j# ||« }
|
|||
|
|
| j j% ||« } | t' |« z } t) | « |
|
|||
|
|
fS )NzEntityLinker_v1.get_lossrk Trl r„ z gold entities do not match up©ÚmethodÚmsg)r rs rq rt ru r@ Ú
|
|||
|
|
get_vectorr[ r2 r] Ú asarray2fÚshaper ÚE147rI ry r? Úget_gradr„ rO Úfloat) rA r) ro Úentity_encodingsr‡ rŠ r‹ rŒ Úentity_encodingÚerrÚ gradientsr” s r+ r„ zEntityLinker_v1.get_lossÌ s" € ܘ(Ð$>Ô?Ø ÐÛˆBØ—^‘^ K¸4�^Ó@ˆFØ—|‘|×(Ô(�Ø ˜sŸy™yÑ)�ÚØ&*§g¡g×&8Ñ&8¸Ó&?�OØ$×+Ñ+¨OÕ<ñ )ð ð Ÿ:™:Ÿ>™>×3Ñ3Ð4DÓEÐØ × #Ñ #Ð'7×'=Ñ'=Ò =Ü—+‘+×$Ñ$Ø!Ð'Fð %ó ˆCô ˜sÓ#Ð #Ø—M‘M×*Ñ*Ð+=Ð?OÓPˆ Ø�}‰}×%Ñ%Ð&8Ð:JÓKˆØ”cÐ*Ó+Ñ+ˆÜ�T‹{˜IÐ%Ð%r- Údocsc ó — | j « d}g }|s|S t |t « r|g}t |« D �]ƒ \ }}|j D �cg c] }|‘Œ }}t |« dkD sŒ/|j D �]E }|j } |j | « }
|
|||
|
|
|
|
|||
|
|
dk\ sJ ‚t d|
|
|||
|
|
| j z
|
|||
|
|
« } t t |« dz
|
|||
|
|
|
|
|||
|
|
| j z « } || j }
|
|||
|
|
|| j }||
|
|||
|
|
| j « }| j j j" }| j$ rF| j j' |g« d }|j( }|j* j- |« }|dz
|
|||
|
|
}|j. | j0 v r |j3 | j4 « �ŒCt7 | j9 | j: |« « }|s |j3 | j4 « �Œ‡t |« dk( r |j3 |d j< « �Œµt? j@ |« |jC |D �cg c] }|jD ‘Œ c}« }| jF s |jC |D �cg c] }d‘Œ c}« }|}| j$ rÜ|jC |D �cg c] }|jH ‘Œ c}« }|j* j- |d¬« }t |« t |« k7 r*tK tL jN jQ dd¬« « ‚|jS |« |z z }|jT |jT k7 rtW tL jX « ‚||z ||z z
|
|||
|
|
}|j[ « j] « } || } |j3 | j< « �ŒH �Œ† t |« |k( s,tL jN jQ dd¬« } tK | « ‚|S c c}w c c}w c c}w c c}w ) ap Apply the pipeline's model to a batch of docs, without modifying them.
|
|||
|
|
Returns the KB IDs for each entity in each doc, including NIL if there is
|
|||
|
|
no prediction.
|
|||
|
|
|
|||
|
|
docs (Iterable[Doc]): The documents to predict.
|
|||
|
|
RETURNS (List[str]): The models prediction for each document.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#predict
|
|||
|
|
r rn rf )ÚaxisÚpredictzvectors not of equal lengthr˜ z$result variables not of equal length)/rR Ú
|
|||
|
|
isinstancer Ú enumeraterr rO rt rw rv r{ r5 r| ru r} r r2 r] Úxpr7 r¨ ÚTÚlinalgÚnormÚlabel_r4 r[ r( r= r9 r@ Úentity_ÚrandomÚshufflera Ú
|
|||
|
|
prior_probr6 Ú
|
|||
|
|
entity_vectorry r rž rI Údotr� rG ÚE161ÚargmaxÚitem) rA r¥ Ú entity_countÚ final_kb_idsÚiÚdocrˆ r‰ r‹ rw r� rŽ r� r� r‘ r’ r« Úsentence_encodingÚsentence_encoding_tÚ
|
|||
|
|
sentence_normÚ
|
|||
|
|
candidatesÚcÚ prior_probsÚ_Úscoresr¡ Ú entity_normÚsimsÚ
|
|||
|
|
best_indexÚbest_candidater£ s r+ r¨ zEntityLinker_v1.predictá s™ € ð
|
|||
|
|
×ÑÔ؈ Ø"$ˆ ÙØ Ð Ü �dœCÔ Ø�6ˆDÜ —o‰FˆAˆsØ$'§I¢IÓ.¡I˜qš IˆIÐ.Ü�3‹x˜!‹|à Ÿ8�8�CØ Ÿ8™8�DØ!*§ ¡ °Ó!6�JØ%¨š?Ð*˜?ä%(¨¨J¸¿ ¹ Ñ,EÓ%F�NÜ#&¤s¨9£~¸Ñ'9¸:ÈÏ É Ñ;TÓ#U�LØ"+¨NÑ";×"AÑ"A�KØ )¨,Ñ 7× ;Ñ ;�IØ " ;¨yÐ 9× @Ñ @Ó B�Hà Ÿ™Ÿ ™ ×*Ñ*�BØ×(Ò(Ø,0¯J©J×,>Ñ,>À¸zÓ,JÈ1Ñ,MÐ)Ø.?×.AÑ.AÐ+Ø(*¯ © ¯©Ð7JÓ(K˜
|
|||
|
|
Ø AÑ%�LØ—z‘z T×%8Ñ%8Ñ8à$×+Ñ+¨D¯H©HÖ5ä%)¨$×*=Ñ*=¸d¿g¹gÀsÓ*KÓ%L˜
|
|||
|
|
Ù )à (× /Ñ /°·±Ö 9Ü ›_°Ò 1à (× /Ñ /°
|
|||
|
|
¸1±
|
|||
|
|
×0EÑ0EÖ Fä "ŸN™N¨:Ô 6à*,¯*©*ÉJÓ5WÉJÀq°a·l³lÈJÑ5WÓ*X˜KØ#'§?¢?Ø.0¯j©jÁzÓ9RÁzÀ!º#ÀzÑ9RÓ.S Ø%0˜Fà #× 0Ò 0Ø35·:±:Ù>HÓ$I¹j¸ Q§_£_¸jÑ$Ió4"Ð 0ð /1¯i©i¯n©nÐ=MÐTU¨nÓ.V Ü#&Ð'7Ó#8¼CÀ Ó<LÒ#LÜ*6Ü(.¯ © ×(:Ñ(:Ø3<Ø0Mð );ó )*ó+&ð %&ð (*§v¡vÐ.>Ð@SÓ'TØ$1°KÑ$?ñ(" ð $(§:¡:°×1BÑ1BÒ#BÜ*4´V·[±[Ó*AÐ$AØ)4°tÑ);¸{ÈTÑ?QÑ)R Ø)/¯ © « ×)=Ñ)=Ó)?˜JØ-7¸
|
|||
|
|
Ñ-C˜NØ (× /Ñ /°×0FÑ0FÖ Gòs $ð &ô| �LÓ! \Ò1Ü—+‘+×$Ñ$Ø Ð&Lð %ó ˆCô ˜sÓ#Ð #ØÐùòE /ùòD 6Xùâ9Rùò
|
|||
|
|
%Js Á O:É O? Ê P Ê-P rŠ c ó� — t |D ��cg c] }|j D ] }|‘Œ Œ c}}« }|t |« k7 r3t t j j |t |« ¬« « ‚d}| j d }|D ]= }|j D ], }|| }|dz
|
|||
|
|
}|D ] } | j dk( s|sŒ|| _ Œ Œ. Œ? yc c}}w )a Modify a batch of documents, using pre-computed scores.
|
|||
|
|
|
|||
|
|
docs (Iterable[Doc]): The documents to modify.
|
|||
|
|
kb_ids (List[str]): The IDs to set, produced by EntityLinker.predict.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#set_annotations
|
|||
|
|
)rt Úidsr r/ rn N) rO rt rG r ÚE148rI r> Ú ent_kb_idÚ
|
|||
|
|
ent_kb_id_)
|
|||
|
|
rA r¥ rŠ r¼ r‹ Ú
|
|||
|
|
count_entsr» r/ rŒ Útokens
|
|||
|
|
r+ Úset_annotationsz EntityLinker_v1.set_annotations7 s» € ô ©ÔB© #¸¿¼°#š#¸˜#¨ÒBÓCˆ
|
|||
|
|
Ø œ˜V›Ò $Ü œVŸ[™[× /Ñ /°ZÄSÈÃ[Ð /Ó QÓRÐ RØ
|
|||
|
|
ˆØ—H‘H˜[Ñ)ˆ ÛˆCØ—x”x�Ø ˜q™ �Ø�Q‘�Û �EØ —‘¨!Ò+ªyØ+0˜Õ(ñ !ñ ñ ùó Cs ‹C
|
|||
|
|
©Úexcludec ó ‡ ‡— ‰ j « i }t ‰ d« r‰ j �ˆ fd„|d< ˆˆ fd„|d< ‰ j j |d< ‰ j
|
|||
|
|
j |d< t
|
|||
|
|
j |‰« S )zéSerialize the pipe to a bytestring.
|
|||
|
|
|
|||
|
|
exclude (Iterable[str]): String names of serialization fields to exclude.
|
|||
|
|
RETURNS (bytes): The serialized object.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/entitylinker#to_bytes
|