178 lines
15 KiB
Plaintext
178 lines
15 KiB
Plaintext
|
Ë
|
|||
|
|
>û g-1 ã ó — d dl mZmZmZmZmZmZmZ d dlm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z d dl
|
|||
|
|
mZ ddlmZ ddlmZ ddlmZ ddlmZmZ dd lmZ dd
|
|||
|
|
lmZ d d l m Z d d
|
|||
|
|
l m Z dZ e « jC e « d Z" ejF ddgde"e ddddidœde › d�dde › d�d de › d �d i¬ « d ed e$de
|
|||
|
|
ee ef d!e$d"e%d#ee& d$ee& d%ee d&d'fd(„« Z' ejP d« d)„ « Z)d*ee d&ee$ef fd+„Z*d,ed&ee&e&f fd-„Z+ G d.„ d'e « Z,y)/é )ÚAnyÚCallableÚDictÚIterableÚListÚOptionalÚTuple)ÚConfigÚModelÚ OptimizerÚset_dropout_rate)ÚFloats2dé )ÚErrors)ÚLanguage)ÚScorer)ÚDocÚSpan)ÚExample)Úregistryé )ÚDEFAULT_SPANS_KEY)Ú
|
|||
|
|
TrainablePipeaó
|
|||
|
|
[model]
|
|||
|
|
@architectures = "spacy.SpanFinder.v1"
|
|||
|
|
|
|||
|
|
[model.scorer]
|
|||
|
|
@layers = "spacy.LinearLogistic.v1"
|
|||
|
|
nO = 2
|
|||
|
|
|
|||
|
|
[model.tok2vec]
|
|||
|
|
@architectures = "spacy.Tok2Vec.v2"
|
|||
|
|
|
|||
|
|
[model.tok2vec.embed]
|
|||
|
|
@architectures = "spacy.MultiHashEmbed.v2"
|
|||
|
|
width = 96
|
|||
|
|
rows = [5000, 1000, 2500, 1000]
|
|||
|
|
attrs = ["NORM", "PREFIX", "SUFFIX", "SHAPE"]
|
|||
|
|
include_static_vectors = false
|
|||
|
|
|
|||
|
|
[model.tok2vec.encode]
|
|||
|
|
@architectures = "spacy.MaxoutWindowEncoder.v2"
|
|||
|
|
width = ${model.tok2vec.embed.width}
|
|||
|
|
window_size = 1
|
|||
|
|
maxout_pieces = 3
|
|||
|
|
depth = 4
|
|||
|
|
ÚmodelÚ span_finderz doc.spansç à?é Nz@scorerszspacy.span_finder_scorer.v1)Ú thresholdr Ú spans_keyÚ
|
|||
|
|
max_lengthÚ
|
|||
|
|
min_lengthÚscorerÚspans_Ú_fg ð?Ú_pç Ú_r)ÚassignsÚdefault_configÚdefault_score_weightsÚnlpÚnamer r r r! r" ÚreturnÚ
|
|||
|
|
SpanFinderc
|
|||
|
|
ó( — t | |||||||¬« S )aÀ Create a SpanFinder component. The component predicts whether a token is
|
|||
|
|
the start or the end of a potential span.
|
|||
|
|
|
|||
|
|
model (Model[List[Doc], Floats2d]): A model instance that
|
|||
|
|
is given a list of documents and predicts a probability for each token.
|
|||
|
|
spans_key (str): Key of the doc.spans dict to save the spans under. During
|
|||
|
|
initialization and training, the component will look for spans on the
|
|||
|
|
reference document under the same key.
|
|||
|
|
threshold (float): Minimum probability to consider a prediction positive.
|
|||
|
|
max_length (Optional[int]): Maximum length of the produced spans, defaults
|
|||
|
|
to None meaning unlimited length.
|
|||
|
|
min_length (Optional[int]): Minimum length of the produced spans, defaults
|
|||
|
|
to None meaning shortest span length is 1.
|
|||
|
|
scorer (Optional[Callable]): The scoring method. Defaults to
|
|||
|
|
Scorer.score_spans for the Doc.spans[spans_key] with overlapping
|
|||
|
|
spans allowed.
|
|||
|
|
)r r r, r" r r! r )r. )r+ r, r r r r r! r" s ú[C:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/pipeline/span_finder.pyÚmake_span_finderr1 , s* € ôX Ø ØØØ
|
|||
|
|
ØØ Ø Øô ð ó c ó — t S ©N)Úspan_finder_score© r2 r0 Úmake_span_finder_scorerr7 d s € ä Ð r2 Úexamplesc óR ‡‡— t |« }dŠ|d Š|j d‰› ‰› �« |j dˆfd„« |j dˆfd„« |j dd « |j d
|
|||
|
|
d « t j | fi |¤Ž}|j |d › d �d « |S )
|
|||
|
|
Nr# r ÚattrÚgetterc óT •— | j j |t ‰« d g « S r4 )ÚspansÚgetÚlen)ÚdocÚkeyÚ attr_prefixs €r0 ú<lambda>z#span_finder_score.<locals>.<lambda>o s! ø€ 3§9¡9§=¡=°´S¸Ó5EÐ5GÐ1HÈ"Ô#Mr2 Úhas_annotationc ó •— ‰| j v S r4 )r= )r@ rA s €r0 rC z#span_finder_score.<locals>.<lambda>q s ø€ °C¸3¿9¹9Ñ4Dr2 Ú
|
|||
|
|
allow_overlapTÚlabeledFÚ _per_type)ÚdictÚ
|
|||
|
|
setdefaultr Ú score_spansÚpop)r8 ÚkwargsÚscoresrB rA s @@r0 r5 r5 i s° ù€ Ü
|
|||
|
|
�&‹\€FØ€KØ
|
|||
|
|
�Ñ
|
|||
|
|
€CØ
|
|||
|
|
×Ñ�f
|
|||
|
|
¨c¨UÐ 3Ô4Ø
|
|||
|
|
×ÑØÓMôð ×ÑÐ&Ó(DÔEØ
|
|||
|
|
×Ñ�o tÔ,Ø
|
|||
|
|
×Ñ�i Ô'Ü
|
|||
|
|
×
|
|||
|
|
Ñ
|
|||
|
|
Ñ
|
|||
|
|
3¨FÑ
|
|||
|
|
3€FØ
|
|||
|
|
‡J�J�&˜‘.Ð! Ð+¨TÔ2Ø €Mr2 Úspanc ód — | d j }| d j t | d « z }||fS )Nr éÿÿÿÿ)Úidxr? )rO ÚstartÚends r0 Ú
|
|||
|
|
_char_indicesrU y s3 € Ø �‰G�K‰K€EØ
|
|||
|
|
ˆr‰(�,‰,œ˜T "™X› Ñ
|
|||
|
|
&€CØ �#ˆ:Ðr2 c óD — e Zd ZdZ d edddedœdedeee e
|
|||
|
|
f de de d e d
|
|||
|
|
e
|
|||
|
|
e d e
|
|||
|
|
e d e
|
|||
|
|
e d
|
|||
|
|
dfd„Zdee fd„Zdee de
|
|||
|
|
d
|
|||
|
|
dfd„Zddddœdee de de
|
|||
|
|
e de
|
|||
|
|
ee e f d
|
|||
|
|
ee e f f
|
|||
|
|
d„Zd
|
|||
|
|
ee e
|
|||
|
|
f fd„Zd
|
|||
|
|
ee
|
|||
|
|
e
|
|||
|
|
f fd„Zdd œd eg ee f de
|
|||
|
|
e d
|
|||
|
|
dfd „Zy) r. zUPipeline that learns span boundaries.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder
|
|||
|
|
r N)r r r r! r" r+ r r, r r r r! r" r- c óØ — |j | _ |�|dk s|�/|dk r*t t j j ||¬« « ‚|| _ || _ || _ ||||dœ| _ y)a Initialize the span finder.
|
|||
|
|
model (thinc.api.Model): The Thinc Model powering the pipeline
|
|||
|
|
component.
|
|||
|
|
name (str): The component instance name, used to add entries to the
|
|||
|
|
losses during training.
|
|||
|
|
threshold (float): Minimum probability to consider a prediction
|
|||
|
|
positive.
|
|||
|
|
scorer (Optional[Callable]): The scoring method.
|
|||
|
|
spans_key (str): Key of the doc.spans dict to save the spans under.
|
|||
|
|
During initialization and training, the component will look for
|
|||
|
|
spans on the reference document under the same key.
|
|||
|
|
max_length (Optional[int]): Maximum length of the produced spans,
|
|||
|
|
defaults to None meaning unlimited length.
|
|||
|
|
min_length (Optional[int]): Minimum length of the produced spans,
|
|||
|
|
defaults to None meaning shortest span length is 1.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder#init
|
|||
|
|
Nr )r! r )r! r r r ) ÚvocabÚ
|
|||
|
|
ValueErrorr ÚE1053Úformatr r, r" Úcfg) Úselfr+ r r, r r r r! r" s r0 Ú__init__zSpanFinder.__init__… sw € ð< —Y‘YˆŒ
|
|||
|
|
Ø Ð " z°A¢~Ø Ð " z°A¢~ä Ü— ‘ ×#Ñ#¨zÀjÐ#ÓQóð
|
|||
|
|
𠈌
|
|||
|
|
؈Œ Ø ˆŒ à$Ø$Ø"Ø"ñ $
|
|||
|
|
ˆ�r2 Údocsc ó< — | j j |« }|S )a Apply the pipeline's model to a batch of docs, without modifying
|
|||
|
|
them.
|
|||
|
|
|
|||
|
|
docs (Iterable[Doc]): The documents to predict.
|
|||
|
|
RETURNS: The models prediction for each document.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder#predict
|
|||
|
|
)r Úpredict)r] r_ rN s r0 ra zSpanFinder.predict´ s € ð —‘×#Ñ# DÓ)ˆØˆ
|
|||
|
|
r2 rN c ó¼ — d}t |« D �]K \ }}g |j | j d < g }g }|||t |« z }t ||« D ]f \ } }
|
|||
|
|
|
|
|||
|
|
d | j d k\ r|j | j « |
|
|||
|
|
d | j d k\ sŒL|j | j « Œh |D ]Œ } |D ]… } | dz | z
|
|||
|
|
}
|
|||
|
|
|
|
|||
|
|
dk rŒ| j d �| j d |
|
|||
|
|
k sŒ3| j d �|
|
|||
|
|
| j d k sŒU|j | j d j || | dz « Œ‡ ŒŽ |t |« z
|
|||
|
|
}�ŒN y)a Modify a batch of Doc objects, using pre-computed scores.
|
|||
|
|
docs (Iterable[Doc]): The documents to modify.
|
|||
|
|
scores: The scores to set, produced by SpanFinder predict method.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder#set_annotations
|
|||
|
|
r r r r r! Nr )Ú enumerater= r\ r? ÚzipÚappendÚi)r] r_ rN Úoffsetrf r@ ÚstartsÚendsÚ
|
|||
|
|
doc_scoresÚtokenÚ token_scorerS rT Ú span_lengths r0 Úset_annotationszSpanFinder.set_annotationsÀ sZ € ð ˆÜ —o‰FˆAˆsØ/1ˆC�I‰I�d—h‘h˜{Ñ+Ñ ,؈F؈DØ ¨´#°c³(Ñ):Ð;ˆJä&)¨#¨zÖ&:Ñ"��{Ø ˜q‘> T§X¡X¨kÑ%:Ò:Ø—M‘M %§'¡'Ô*Ø ˜q‘> T§X¡X¨kÑ%:Ó:Ø—K‘K §¡Õ(ð ';ó �Û �CØ"%¨¡'¨E¡/�KØ" Q’Ø à Ÿ™ Ñ.Ð6Ø Ÿ8™8 LÑ1°[Ó@à Ÿ™ Ñ.Ð6Ø&¨$¯(©(°<Ñ*@Ó@àŸ ™ $§(¡(¨;Ñ"7Ñ8×?Ñ?ÀÀEÈCÐRSÉGÐ@TÕUñ ð ð
|
|||
|
|
”c˜#“hÑ ŠFñ3 &r2 r& )ÚdropÚsgdÚlossesr8 ro rp rq c óx — |€i }|j | j d« |D �cg c] }|j ‘Œ }}t | j |« | j j |« \ }}| j
|
|||
|
|
||« \ } }
|
|||
|
|
||
|
|||
|
|
« |�| j |« || j xx | z
|
|||
|
|
cc< |S c c}w )a? Learn from a batch of documents and gold-standard information,
|
|||
|
|
updating the pipe's model. Delegates to predict and get_loss.
|
|||
|
|
examples (Iterable[Example]): A batch of Example objects.
|
|||
|
|
drop (float): The dropout rate.
|
|||
|
|
sgd (Optional[thinc.api.Optimizer]): The optimizer.
|
|||
|
|
losses (Optional[Dict[str, float]]): Optional record of the loss during
|
|||
|
|
training. Updated using the component name as the key.
|
|||
|
|
RETURNS (Dict[str, float]): The updated losses dictionary.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder#update
|
|||
|
|
r& )rJ r, Ú predictedr
|
|||
|
|
r Ú begin_updateÚget_lossÚ
|
|||
|
|
finish_update) r] r8 ro rp rq Úegrs rN Úbackprop_scoresÚlossÚd_scoress r0 ÚupdatezSpanFinder.updateã s® € ð& ˆ>؈FØ×ј$Ÿ)™) SÔ)Ù,4Ó5©H b�R—\“\¨Hˆ Ð5ܘŸ™ TÔ*Ø"&§*¡*×"9Ñ"9¸)Ó"DÑ ˆ�Ø Ÿ ™ x°Ó8‰ˆˆhÙ˜Ô!Ø ˆ?Ø × Ñ ˜sÔ #؈t�y‰yÓ˜TÑ!Ó؈
|
|||
|
|
ùò 6s ¥B7c óî — | j || j j « \ }}|| j j j |« z
|
|||
|
|
}||z }t |dz j « « }||fS )ab Find the loss and gradient of loss for the batch of documents and
|
|||
|
|
their predicted scores.
|
|||
|
|
examples (Iterable[Examples]): The batch of examples.
|
|||
|
|
scores: Scores representing the model's predictions.
|
|||
|
|
RETURNS (Tuple[float, Floats2d]): The loss and the gradient.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/spanfinder#get_loss
|
|||
|
|
r )Ú_get_aligned_truth_scoresr ÚopsÚ asarray2fÚfloatÚsum)r] r8 rN ÚtruthsÚmasksrz ry s r0 ru zSpanFinder.get_loss sj € ð ×6Ñ6°xÀÇÁÇ Á ÓP‰
|
|||
|
|
ˆ�ؘDŸJ™JŸN™N× 4Ñ 4°VÓ <Ñ<ˆØ�EшÜ�h ‘k×&Ñ&Ó(Ó)ˆØ�Xˆ~Ð r2 c ó — g }g }|D �]¸ }|j j |j j k7 r)t t j
|
|||
|
|
j
|
|||
|
|
d¬« « ‚t |j « }|j j |dfd¬« }|j j |dfd¬« }| j d |j j v rÅ|j j | j d D ]œ } t | « \ }
|
|||
|
|
} |j j! |
|
|||
|
|
| d¬« } t | « \ }
|
|||
|
|
}|
|
|||
|
|
|
|
|||
|
|
k( }|| k( }|rd || d
|
|||
|
|
j" d
|
|||
|
|
f< nd
|