917 lines
117 KiB
Plaintext
917 lines
117 KiB
Plaintext
|
Ë
|
|||
|
|
=û gœª ã ód — d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlm Z m
|
|||
|
|
Z
|
|||
|
|
d dl m Z d dl
|
|||
|
|
mZ d dlmZmZ d dlmZ d dlmZ d dlmZmZmZmZmZm Z m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z& d dl'Z'd dl(Z(d d l)m*Z* d d
|
|||
|
|
l+m,Z,m-Z-m.Z.m/Z/ d d l0m1Z1 d d
|
|||
|
|
l2m3Z3m4Z4m5Z5 d dl6m7Z7 d dl8m9Z9m:Z: d dl;m<Z< d dl=m>Z>m?Z?m@Z@ d dlAmBZBmCZC d dlDmEZE d dlFmGZGmHZHmIZI d dlJmKZKmLZLmMZMmNZNmOZO d dlPmQZQ d dlRmSZS d dlTmUZU d dlVmWZW d dlXmYZYmZZZ d dl[m\Z\m]Z] d d l5m^Z^m_Z_m`Z`maZambZbmcZcmdZdmeZemfZf d d lgmhZh d d limjZjmkZk eeUgeUf Zl eem« jÜ d z Zo e5jà eo« Zq eem« jÜ d z Zr e#d!« Zs G d"„ d#« Zt eejê d$« d%ed&geSf fd'„« Zv eejî d(« d)„ « Zx G d*„ d&« Zyd+ezd%ezfd,„Z{e G d-„ d.« « Z| G d/„ d0e}« Z~d1eeY d%e eY fd2„Zd3ee$e€eUe�f esgeUf d4eed5eeU f d6e"ezezezf d%dfd7„Z‚ G d8„ d9« Zƒ G d:„ d;« Z„ e„« Z…y)<é N)Ú ExitStackÚcontextmanager)Údeepcopy)Ú dataclass)ÚchainÚcycle)ÚPath)Ú
|
|||
|
|
default_timer)ÚAnyÚCallableÚDictÚIterableÚIteratorÚListÚNoReturnÚOptionalÚPatternÚSequenceÚSetÚTupleÚTypeVarÚUnionÚcastÚoverload)ÚPool)ÚConfigÚCupyOpsÚ OptimizerÚget_current_ops)Úconvert_recursiveé )ÚaboutÚtyÚutil)ÚLiteral)ÚErrorsÚWarnings)Ú GIT_VERSION)ÚTOKENIZER_INFIXESÚTOKENIZER_PREFIXESÚTOKENIZER_SUFFIXES)ÚBASE_EXCEPTIONSÚ URL_MATCH)Ú load_lookups)Ú
|
|||
|
|
analyze_pipesÚprint_pipe_analysisÚvalidate_attrs)Ú ConfigSchemaÚConfigSchemaInitÚConfigSchemaNlpÚConfigSchemaPretrainÚvalidate_init_settings)ÚScorer©Ú Tokenizer)ÚDoc)Ú
|
|||
|
|
Underscore)ÚExampleÚvalidate_examples)Ú init_tok2vecÚ
|
|||
|
|
init_vocab) Ú_DEFAULT_EMPTY_PIPESÚCONFIG_SECTION_ORDERÚSimpleFrozenDictÚSimpleFrozenListÚ_pipeÚcombine_score_weightsÚ raise_errorÚregistryÚwarn_if_jupyter_cupy)Ú BaseVectors)ÚVocabÚ create_vocabzdefault_config.cfgz default_config_pretraining.cfgÚ _AnyContextc óF — e Zd ZU dZ ee¬« Zeed< eZ e
|
|||
|
|
e e e
|
|||
|
|
f ed< eZeeee ef ed< eZeeee ef ed< eZeeee ef ed< dZee ed < eZee ed
|
|||
|
|
< i Z e
|
|||
|
|
e ef ed < i Z e
|
|||
|
|
e ee ge f f ed < e « Z!e"e ed
|
|||
|
|
< ddddœZ#y)Ú BaseDefaultsz¦Language data defaults, available via Language.Defaults. Can be
|
|||
|
|
overwritten by language subclasses by defining their own subclasses of
|
|||
|
|
Language.Defaults.
|
|||
|
|
©Ú
|
|||
|
|
section_orderÚconfigÚtokenizer_exceptionsÚprefixesÚsuffixesÚinfixesNÚ token_matchÚ url_matchÚsyntax_iteratorsÚlex_attr_gettersÚ
|
|||
|
|
stop_wordsÚltrT)Ú directionÚhas_caseÚ has_letters)$Ú__name__Ú
|
|||
|
|
__module__Ú __qualname__Ú__doc__r rA rQ Ú__annotations__r, rR r
|
|||
|
|
Ústrr Údictr* rS r r r r r+ rT r) rU rV r r- rW rX rY Úintr ÚsetrZ r Úwriting_system© ó úOC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/language.pyrN rN X sö … ññ
|
|||
|
|
Ð*>Ô?€FˆFÓ?Ø2AИ$˜s D¨¡J˜Ñ/ÓAØ8J€Hˆh�x c¨7 lÑ 3Ñ4Ñ5ÓJØ8J€Hˆh�x c¨7 lÑ 3Ñ4Ñ5ÓJØ7H€GˆX�h˜u S¨' \Ñ 2Ñ3Ñ
|
|||
|
|
4ÓHØ&*€K�˜(Ñ#Ó*Ø$-€Iˆx˜Ñ!Ó-Ø,.Ð�d˜3 ˜=Ñ)Ó.Ø8:Ð�d˜3 ¨#¨°¨Ñ 4Ð4Ñ5Ó:Ù ›5€J��C‘Ó Ø#(°dÈ4ÑP�Nrj rN zspacy.Tokenizer.v1ÚreturnÚLanguagec ó — dddt fd„} | S )z�Registered function to create a tokenizer. Returns a factory that takes
|
|||
|
|
the nlp object and returns a Tokenizer instance using the language detaults.
|
|||
|
|
Únlprm rl c ó — | j j }| j j }| j j }|r t j
|
|||
|
|
|« j nd }|r t j |« j nd }|r t j |« j nd }t | j | j j |||| j j | j j ¬« S )N)ÚrulesÚ
|
|||
|
|
prefix_searchÚ
|
|||
|
|
suffix_searchÚinfix_finditerrV rW )ÚDefaultsrS rT rU r$ Úcompile_prefix_regexÚsearchÚcompile_suffix_regexÚcompile_infix_regexÚfinditerr9 ÚvocabrR rV rW )ro rS rT rU rr rs rt s rk Útokenizer_factoryz+create_tokenizer.<locals>.tokenizer_factoryq sÄ € Ø—<‘<×(Ñ(ˆØ—<‘<×(Ñ(ˆØ—,‘,×&Ñ&ˆÙFNœ×1Ñ1°(Ó;×BÒBÐTXˆ
|
|||
|
|
ÙFNœ×1Ñ1°(Ó;×BÒBÐTXˆ
|
|||
|
|
ÙGNœ×1Ñ1°'Ó:×CÒCÐTXˆÜØ �I‰IØ—,‘,×3Ñ3Ø'Ø'Ø)ØŸ ™ ×0Ñ0Ø—l‘l×,Ñ,ô
|
|||
|
|
ð
|
|||
|
|
rj r8 )r| s rk Úcreate_tokenizerr} k s € ð
|
|||
|
|
˜zð
|
|||
|
|
¬ió
|
|||
|
|
ð" Ð rj zspacy.LookupsDataLoader.v1c ó` — t j j d|« t | |¬« }|S )Nz+Loading lookups from spacy-lookups-data: %s)ÚlangÚtables)r$ ÚloggerÚdebugr. )r r€ Úlookupss rk Úload_lookups_datar„ … s( € ä‡K�K×ÑÐCÀVÔLÜ ¨VÔ4€GØ €Nrj c óü
|
|||
|
|
‡ — e Zd ZU dZeZdZee e d< e
|
|||
|
|
Z e e
|
|||
|
|
j ¬« Zi Zeedf e d< dŸdi ddd d
|
|||
|
|
œd eeef d ed
|
|||
|
|
eeef deed geegef f deedgef deddfd„Zˆ fd„Ze d„ « Z e deeef fd„« Z e j> deeef ddfd„« Z e de fd„« Z!e!j> de ddfd„« Z!e de"e fd„« Z#e de"e fd „« Z$e de"e%ee&f fd „« Z'e de"e fd „« Z(e de"e%ee&f fd „« Z)e de"e fd „« Z*e deeef fd!„« Z+e deee"e f fd"„« Z,e-d#edefd$„« Z.e-d#edefd%„« Z/e-d#eddfd&„« Z0e-d#eddddfd'„« Z1d#eddfd(„Z2d#ede fd)„Z3e- e « e4« e4« d* e « dd+œd#ed,eeef d-e5e d.e5e d/ed0eeee6 f d1ee defd2„« Z7e- e4« e4« d*dd3œd#ed-e5e d.e5e d/ed1ee& ded4ef f d5„« Z8g d6¢d*d7œd8e"e d9edeeeef fd:„Z9d#ede&fd;„Z: d e « ddd<œd=ed#ee d>eeef d?ee d@ede&f dA„Z;dBedCd d#ede%e&ef fdD„Z< d ddddd e « dddEœd=ed#ee dFeeeef dGeeeef dHee dIee dCed d>eeef d?ee d@ede&fdJ„Z= d¡dFeeeef dGeeeef dHee dIee def
|
|||
|
|
dK„Z>d#edefdL„Z? e « ddMœd#ed=ed>eeef d@ede&f
|
|||
|
|
dN„Z@dOedPeddfdQ„ZAd#ede%ee&f fdR„ZBd#eddfdS„ZCd#eddfdT„ZD e4« ddUœdVeeef dWe5e dXeeeeeef f defdY„ZEd¢d[„ZFddd\œdWeeee5e f d]eeee5e f ddZfd^„ZGdVedefd_„ZHd`eeeeIf defda„ZJd`eeeeIf dbeKdefdc„ZL d ddddd e4« e4« deœdfe5eM dgee dhe6dieeN djeeee6f dXeeeeeef f dke5e dle5e fdm„ZOddd e4« dnœdfe5eM dieeN djeeee6f dXeeeeeef f dke5e deee6f f do„ZP d ddpœdqeeg e5eM f dieeN deNfdr„ZQ d ddpœdqeeg e5eM f dieeN deNfds„ZRddpœdieeN deNfdt„ZSdueee&e"e eTgeUf fdv„ZVddddd*dwœdfe5eM dee dxeeW dXeeeeeef f dyeeeef dzedeeef fd{„ZXd|„ ZYeZd}ee[ fd~„« Z\e]d4d4d4d4d4dœd€e5eeef d�e^d* dee dWe5e dXeeeeeef f d‚ede_e fdƒ„« Z`e]d4d4d4d4d4dœd€e5e%eeef eKf d�e^d dee dWe5e dXeeeeeef f d‚ede_e%eeKf fd„„« Z`d*d e4« dd…dœd€ee5eeef e5e%eeef eKf f d�edee dWe5e dXeeeeeef f d‚edee_e e_e%eeKf f fd†„Z`dWe5e fd‡„Zad€e5eeef dˆe5ed4e_e f d‚edede_e f
|
|||
|
|
d‰„Zbd£dŠ„Zce-i fdededed e « ddd‹œd>eeeef e f d eeef dWeee5e f d]eee5e f dkeee5e f d
|
|||
|
|
eeef dŒed@edd fd�„« ZedŽed�ed�e5e ddfd‘„ZfeZd d’eeg de_eg fd“„« Zh e4« d”œd•eeeif dke5e ddfd–„ZjekdWeee5e f d]eee5e f d—e5e de%ed4f fd˜„« Zl e4« e « d™œd•eeeif dke5e dšeeef dd fd›„Zm e4« d”œdke5e deIfdœ„Zn e4« d”œd�eIdke5e dd fdž„Zoˆ xZpS )¤rm aR A text-processing pipeline. Usually you'll load this once per process,
|
|||
|
|
and pass the instance around your application.
|
|||
|
|
|
|||
|
|
Defaults (class): Settings, data and factory methods for creating the `nlp`
|
|||
|
|
object and processing pipeline.
|
|||
|
|
lang (str): IETF language code, such as 'en'.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language
|
|||
|
|
Nr ©ÚerrorÚ FactoryMetaÚ
|
|||
|
|
_factory_metaTi@B iè )Ú
|
|||
|
|
max_lengthÚmetar} Úcreate_vectorsÚ
|
|||
|
|
batch_sizer{ rŠ r‹ r} rŒ rJ r� rl c óž — t j j j « t j | j « | _ t |« | _ d| _
|
|||
|
|
d| _ i | _ i | _
|
|||
|
|
t |t « s;|dur7t! t" j$ j' |t) t « ¬« « ‚|du r|j+ di « j+ d« }t- | j. | j0 |¬« }|s,d| j d d i} t j2 | « d } ||« |_ no| j. rc|j. rW| j. |j. k7 r>t! t" j6 j' | j. |j. ¬« « ‚|| _ | j. €| j8 j. | _ g | _ t= « | _ || _ |s,d | j d d i}
|
|||
|
|
t j2 |
|
|||
|
|
« d } || « | _! || _" tF | _$ y)
|
|||
|
|
a Initialise a Language object.
|
|||
|
|
|
|||
|
|
vocab (Vocab): A `Vocab` object. If `True`, a vocab is created.
|
|||
|
|
meta (dict): Custom meta data for the Language class. Is written to by
|
|||
|
|
models to add model meta data.
|
|||
|
|
max_length (int): Maximum number of characters in a single text. The
|
|||
|
|
current models may run out memory on extremely long texts, due to
|
|||
|
|
large internal allocations. You should segment these texts into
|
|||
|
|
meaningful units, e.g. paragraphs, subsections etc, before passing
|
|||
|
|
them to spaCy. Default maximum length is 1,000,000 charas (1mb). As
|
|||
|
|
a rule of thumb, if all pipeline components are enabled, spaCy's
|
|||
|
|
default models currently requires roughly 1GB of temporary memory per
|
|||
|
|
100,000 characters in one text.
|
|||
|
|
create_tokenizer (Callable): Function that takes the nlp object and
|
|||
|
|
returns a tokenizer.
|
|||
|
|
batch_size (int): Default batch size for pipe and evaluate.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#init
|
|||
|
|
NT)r{ Ú
|
|||
|
|
vocab_typeÚvectorsÚname)Ú vectors_namero )ro r{ Ú tokenizer)%r$ rG Ú_entry_point_factoriesÚget_allÚDEFAULT_CONFIGÚmergeÚdefault_configÚ_configre Ú_metaÚ_pathÚ
|
|||
|
|
_optimizerÚ
|
|||
|
|
_pipe_metaÚ
|
|||
|
|
_pipe_configsÚ
|
|||
|
|
isinstancerJ Ú
|
|||
|
|
ValueErrorr& ÚE918ÚformatÚtypeÚgetrK r ru Úresolver� ÚE150r{ Ú _componentsrg Ú _disabledrŠ r“ r� rF Údefault_error_handler) Úselfr{ rŠ r‹ r} rŒ r� Úkwargsr’ Ú vectors_cfgÚ
|
|||
|
|
tokenizer_cfgs rk Ú__init__zLanguage.__init__ž sÊ € ôB
|
|||
|
|
�
|
|||
|
|
‰
|
|||
|
|
×,Ñ,×4Ñ4Ô6ä%×+Ñ+¨D×,?Ñ,?Ó@ˆŒ ܘ$“ZˆŒ
|
|||
|
|
؈Œ
|
|||
|
|
Ø/3ˆŒà46ˆŒØ02ˆÔä˜%¤Ô'¨E¸Ñ,=Ü œVŸ[™[× /Ñ /°eÌÌUË Ð /Ó TÓUÐ UØ �D‰=Ø Ÿ8™8 I¨rÓ2×6Ñ6°vÓ>ˆLÜ §¡¨D¯M©MÈ ÔUˆEÙ!Ø (¨$¯,©,°uÑ*=¸iÑ*HÐ I� Ü!)×!1Ñ!1°+Ó!>¸yÑ!I�Ù *¨5Ó 1ˆE�Mà— ’ ˜eŸjšj¨t¯y©y¸E¿J¹JÒ/FÜ ¤§¡×!3Ñ!3¸¿ ¹ ÈÏÉÐ!3Ó!TÓUÐUØ !ˆŒ
|
|||
|
|
Ø �9‰9Ð Ø Ÿ
|
|||
|
|
™
|
|||
|
|
Ÿ™ˆDŒIØ;=ˆÔÜ#&£5ˆŒØ$ˆŒá Ø (¨$¯,©,°uÑ*=¸kÑ*JÐ KˆMÜ '× /Ñ /°
|
|||
|
|
Ó >¸{Ñ KÐ Ù)¨$Ó/ˆŒØ$ˆŒÜ%0ˆÕ"rj c ó¸ •— t ‰| � di |¤Ž t j | j j
|
|||
|
|
« | _ | j | j d d< y )Nro r ri )ÚsuperÚ__init_subclass__r– r— ru rQ r˜ r )Úclsr« Ú __class__s €rk r± zLanguage.__init_subclass__ã sI ø€ Ü
|
|||
|
|
‰Ñ!Ñ+ FÒ+Ü +× 1Ñ 1°#·,±,×2EÑ2EÓ FˆÔØ,/¯H©Hˆ×ј5Ñ! &Ò)rj c ó — | j S ©N)r› ©rª s rk Úpathz
|
|||
|
|
Language.pathè s € à�z‰zÐrj c óŠ — t j t j « }| j j
|
|||
|
|
r1| j j d| j j
|
|||
|
|
« n&| j j d| j
|
|||
|
|
« | j j dd« | j j dd« | j j d|« | j j dd« | j j d d« | j j d
|
|||
|
|
d« | j j d d« | j j d d« | j j d
|
|||
|
|
t « | j j t | j j « | j j j | j j j | j j j dœ| j d< t | j « | j d< t# | j$ « | j d< t# | j& « | j d< t# | j( « | j d< | j S )zßCustom meta data of the language class. If a model is loaded, this
|
|||
|
|
includes details from the model's meta.json.
|
|||
|
|
|
|||
|
|
RETURNS (Dict[str, Any]): The meta.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#meta
|
|||
|
|
r r‘ ÚpipelineÚversionz0.0.0Ú
|
|||
|
|
spacy_versionÚ descriptionÚ ÚauthorÚemailÚurlÚlicenseÚspacy_git_version)Úwidthr� Úkeysr‘ Úmoder� ÚlabelsÚ
|
|||
|
|
componentsÚdisabled)r$ Úget_minor_version_ranger" Ú __version__r{ r rš Ú
|
|||
|
|
setdefaultr( Úvectors_lengthÚlenr� Ún_keysr‘ rÅ re Ú pipe_labelsÚlistÚ
|
|||
|
|
pipe_namesÚcomponent_namesrÈ )rª r» s rk r‹ z
|
|||
|
|
Language.metaì sä € ô ×4Ñ4´U×5FÑ5FÓGˆ
|
|||
|
|
Ø �:‰:�?Š?Ø �J‰J× !Ñ ! &¨$¯*©*¯/©/Õ :à �J‰J× !Ñ ! &¨$¯)©)Ô 4Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜f jÔ1Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜i¨Ô1Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜o¨}Ô=Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜m¨RÔ0Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜h¨Ô+Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜g rÔ*Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜e RÔ(Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ ˜i¨Ô,Ø �
|
|||
|
|
‰
|
|||
|
|
× Ñ Ð 1´;Ô?à—Z‘Z×.Ñ.ܘ4Ÿ:™:×-Ñ-Ó.Ø—J‘J×&Ñ&×-Ñ-Ø—J‘J×&Ñ&×+Ñ+Ø—J‘J×&Ñ&×+Ñ+ñ !
|
|||
|
|
ˆ�
|
|||
|
|
‰
|
|||
|
|
�9Ñ ô $ D×$4Ñ$4Ó 5ˆ�
|
|||
|
|
‰
|
|||
|
|
�8Ñ ô "& d§o¡oÓ!6ˆ�
|
|||
|
|
‰
|
|||
|
|
�:Ñ Ü#'¨×(<Ñ(<Ó#=ˆ�
|
|||
|
|
‰
|
|||
|
|
�<Ñ Ü!% d§m¡mÓ!4ˆ�
|
|||
|
|
‰
|
|||
|
|
�:Ñ Ø�z‰zÐrj Úvaluec ó — || _ y rµ )rš ©rª rÓ s rk r‹ z
|
|||
|
|
Language.meta s € àˆ�
|
|||
|
|
rj c óv — | j j di « | j j di « | j | j d d< i }g }| j D ]_ }| j |« }| j |« }d|j i|¥||< |j sŒE|j |j « Œa t | j « | j d d< t | j « | j d d< || j d< | j d j di « }t ||« }|| j d d< t j | j « s3t t j" j% | j ¬ « « ‚| j S )
|
|||
|
|
z÷Trainable config for the current language instance. Includes the
|
|||
|
|
current pipeline components, as well as default training config.
|
|||
|
|
|
|||
|
|
RETURNS (thinc.api.Config): The config.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#config
|
|||
|
|
ro Útrainingr Úfactoryr¹ rÈ rÇ Ú
|
|||
|
|
score_weights©rQ )r™ rË r rÒ Ú
|
|||
|
|
get_pipe_metaÚget_pipe_configrØ Údefault_score_weightsÚappendrÐ rÈ r¤ rE ÚsrslyÚis_json_serializabler r& ÚE961r¢ )rª r¹ rÙ Ú pipe_nameÚ pipe_metaÚ pipe_configÚ prev_weightsÚcombined_score_weightss rk rQ zLanguage.config sz € ð
|
|||
|
|
� ‰ × Ñ rÔ*Ø � ‰ × Ñ
|
|||
|
|
¨BÔ/Ø&*§i¡iˆ� ‰ �UјFÑ#ð ˆØˆ
|
|||
|
|
Ø ×-Ô-ˆIØ ×*Ñ*¨9Ó5ˆIØ ×.Ñ.¨yÓ9ˆKØ#,¨i×.?Ñ.?Ð"OÀ;Ð"OˆH�YÑ Ø×.Ó.Ø ×$Ñ$ Y×%DÑ%DÕEð .ô +/¨t×/CÑ/CÓ*Dˆ� ‰ �UјJÑ'Ü*.¨t¯}©}Ó*=ˆ� ‰ �UјJÑ'Ø%-ˆ� ‰ �\Ñ"ð —|‘| JÑ/×3Ñ3°OÀRÓHˆ Ü!6°}ÀlÓ!SÐ Ø4Jˆ� ‰ �ZÑ Ñ1Ü×)Ñ)¨$¯,©,Ô7Ü œVŸ[™[× /Ñ /°t·|±|Ð /Ó DÓEÐ EØ�|‰|Ðrj c ó — || _ y rµ )r™ rÕ s rk rQ zLanguage.config9 s € à ˆ� rj c ó¾ — | j D ��cg c] \ }}|| j v sŒ|‘Œ }}}t |t j j d¬« ¬« S c c}}w )ziGet the names of all disabled components.
|
|||
|
|
|
|||
|
|
RETURNS (List[str]): The disabled components.
|
|||
|
|
rÈ ©Úattrr† ©r§ r¨ rC r& ÚE926r¢ )rª r‘ Ú_Únamess rk rÈ zLanguage.disabled= sT € ð &*×%5Ò%5ÔPÑ%5™'˜$ ¸ÀÇ Á Ò9O’Ð%5ˆÑPÜ ¬V¯[©[×-?Ñ-?ÀZÐ-?Ó-PÔQÐQùó Qs
|
|||
|
|
�A§Ac ó^ — t | j j « « }t |« S )z_Get names of all available factories.
|
|||
|
|
|
|||
|
|
RETURNS (List[str]): The factory names.
|
|||
|
|
)rÐ Ú factoriesrÄ rC ©rª rî s rk Ú
|
|||
|
|
factory_nameszLanguage.factory_namesH s& € ô �T—^‘^×(Ñ(Ó*Ó+ˆÜ Ó&Ð&rj c ól — t | j t j j d¬« ¬« S )zoGet all (name, component) tuples in the pipeline, including the
|
|||
|
|
currently disabled components.
|
|||
|
|
rÇ ré r† )rC r§ r& rì r¢ r¶ s rk rÇ zLanguage.componentsQ s. € ô
|
|||
|
|
Ø × Ñ ¤F§K¡K×$6Ñ$6¸LÐ$6Ó$Iô
|
|||
|
|
ð
|
|||
|
|
rj c ó — | j D ��cg c] \ }}|‘Œ }}}t |t j j d¬« ¬« S c c}}w )zÂGet the names of the available pipeline components. Includes all
|
|||
|
|
active and inactive pipeline components.
|
|||
|
|
|
|||
|
|
RETURNS (List[str]): List of component name strings, in order.
|
|||
|
|
rÒ ré r† )r§ rC r& rì r¢ ©rª râ rí rî s rk rÒ zLanguage.component_namesZ sJ € ð 04×/?Ò/?Ô@Ñ/?™|˜y¨!’Ð/?ˆÑ@Ü ¬V¯[©[×-?Ñ-?ÐEVÐ-?Ó-WÔXÐXùó Aó � A
|
|||
|
|
c ó — | j D ��cg c] \ }}|| j vsŒ||f‘Œ }}}t |t j j d¬« ¬« S c c}}w )zòThe processing pipeline consisting of (name, component) tuples. The
|
|||
|
|
components are called on the Doc in order as it passes through the
|
|||
|
|
pipeline.
|
|||
|
|
|
|||
|
|
RETURNS (List[Tuple[str, Callable[[Doc], Doc]]]): The pipeline.
|
|||
|
|
r¹ ré r† rë )rª ÚnÚpÚpipess rk r¹ zLanguage.pipelined sX € ð %)×$4Ò$4ÔPÑ$4™D˜A˜q¸ÀÇ Á Ò8O�!�Q’Ð$4ˆÑPÜ ¬V¯[©[×-?Ñ-?ÀZÐ-?Ó-PÔQÐQùó Qs
|
|||
|
|
�A§Ac ó — | j D ��cg c] \ }}|‘Œ }}}t |t j j d¬« ¬« S c c}}w )zƒGet names of available active pipeline components.
|
|||
|
|
|
|||
|
|
RETURNS (List[str]): List of component name strings, in order.
|
|||
|
|
rÑ ré r† )r¹ rC r& rì r¢ rõ s rk rÑ zLanguage.pipe_nameso sD € ð 04¯}ª}Ô=©}™|˜y¨!’¨}ˆÑ=Ü ¬V¯[©[×-?Ñ-?À\Ð-?Ó-RÔSÐSùó >rö c ó€ — i }| j D ]# \ }}| j |« j ||< Œ% t |« S )z—Get the component factories for the available pipeline components.
|
|||
|
|
|
|||
|
|
RETURNS (Dict[str, str]): Factory names, keyed by component names.
|
|||
|
|
)r§ rÛ rØ rB )rª rð râ Úpipes rk Úpipe_factorieszLanguage.pipe_factoriesx sC € ð ˆ Ø #× /Ô /‰OˆI�tØ#'×#5Ñ#5°iÓ#@×#HÑ#HˆI�iÒ ð 0ä Ó*Ð*rj c óÄ — i }| j D ]E \ }}t |d« r|j du rŒ!t |d« sŒ.t |j « ||< ŒG t |« S )zéGet the labels set by the pipeline components, if available (if
|
|||
|
|
the component exposes a labels property and the labels are not
|
|||
|
|
hidden).
|
|||
|
|
|
|||
|
|
RETURNS (Dict[str, List[str]]): Labels keyed by component name.
|
|||
|
|
Ú hide_labelsTrÆ )r§ Úhasattrr rÐ rÆ rB )rª rÆ r‘ rý s rk rÏ zLanguage.pipe_labelsƒ s` € ð ˆØ ×*Ô*‰JˆD�$Ü�t˜]Ô+°×0@Ñ0@ÀDÑ0HØÜ�t˜XÕ&Ü # D§K¡KÓ 0��t’ ð +ô
|
|||
|
|
Ó'Ð'rj r‘ c óp — | j |« }|t j v xs |t j v S )z=RETURNS (bool): Whether a factory of that name is registered.)Úget_factory_namerG rð ©r² r‘ Ú
|
|||
|
|
internal_names rk Ú has_factoryzLanguage.has_factory“ s5 € ð ×,Ñ,¨TÓ2ˆ
|
|||
|
|
Ø”x×)Ñ)Ð)ÒP¨]¼h×>PÑ>PÐ-PÐPrj c ó@ — | j €|S | j › d|› �S )zŸGet the internal factory name based on the language subclass.
|
|||
|
|
|
|||
|
|
name (str): The factory name.
|
|||
|
|
RETURNS (str): The internal factory name.
|
|||
|
|
ú.)r )r² r‘ s rk r zLanguage.get_factory_name™ s( € ð �8‰8Р؈KØ—(‘(�˜1˜T˜FÐ#Ð#rj c óì — | j |« }|| j v r| j | S || j v r| j | S t t j j d|¬« « ‚)z°Get the meta information for a given factory name.
|
|||
|
|
|
|||
|
|
name (str): The component factory name.
|
|||
|
|
RETURNS (FactoryMeta): The meta for the given factory name.
|
|||
|
|
rØ ©r‹ r‘ )r r‰ r r& ÚE967r¢ r s rk Úget_factory_metazLanguage.get_factory_meta¤ sq € ð ×,Ñ,¨TÓ2ˆ
|
|||
|
|
Ø ˜C× -Ñ -Ñ -Ø×$Ñ$ ]Ñ3Ð 3Ø �3×$Ñ$Ñ $Ø×$Ñ$ TÑ*Ð *ÜœŸ™×+Ñ+°ÀÐ+ÓFÓGÐGrj c ó@ — || j | j |« < y)zšSet the meta information for a given factory name.
|
|||
|
|
|
|||
|
|
name (str): The component factory name.
|
|||
|
|
value (FactoryMeta): The meta to set.
|
|||
|
|
N)r‰ r )r² r‘ rÓ s rk Úset_factory_metazLanguage.set_factory_meta² s € ð 9>ˆ×ј#×.Ñ.¨tÓ4Ò5rj c ó� — || j vr*t t j j d|¬« « ‚| j | S )z¬Get the meta information for a given component name.
|
|||
|
|
|
|||
|
|
name (str): The component name.
|
|||
|
|
RETURNS (FactoryMeta): The meta for the given component name.
|
|||
|
|
Ú componentr
|
|||
|
|
)r� r r& r r¢ ©rª r‘ s rk rÛ zLanguage.get_pipe_meta» s> € ð �t—‘Ñ &Ü œVŸ[™[× /Ñ /°[ÀtÐ /Ó LÓMÐ MØ�‰˜tÑ$Ð$rj c ó’ — || j vr)t t j j |¬« « ‚| j | }|S )z±Get the config used to create a pipeline component.
|
|||
|
|
|
|||
|
|
name (str): The component name.
|
|||
|
|
RETURNS (Config): The config used to create the pipeline component.
|
|||
|
|
©r‘ )rž r r& ÚE960r¢ )rª r‘ rä s rk rÜ zLanguage.get_pipe_configÅ sE € ð �t×)Ñ)Ñ )Ü œVŸ[™[× /Ñ /°TÐ /Ó :Ó;Ð ;Ø×(Ñ(¨Ñ.ˆ ØÐrj F)r˜ ÚassignsÚrequiresÚ retokenizesrÝ Úfuncr˜ r r r rÝ r c ó° ‡ ‡‡‡‡‡‡— t ‰t « s)t t j j d¬« « ‚d‰v r)t t j j ‰¬« « ‚t ‰t « s6t j j d‰t ‰« ¬« }t |« ‚dt dt fˆˆ ˆˆˆˆˆfd „ } |� | |« S | S )
|
|||
|
|
a~ Register a new pipeline component factory. Can be used as a decorator
|
|||
|
|
on a function or classmethod, or called as a function with the factory
|
|||
|
|
provided as the func keyword argument. To create a component and add
|
|||
|
|
it to the pipeline, you can use nlp.add_pipe(name).
|
|||
|
|
|
|||
|
|
name (str): The name of the component factory.
|
|||
|
|
default_config (Dict[str, Any]): Default configuration, describing the
|
|||
|
|
default values of the factory arguments.
|
|||
|
|
assigns (Iterable[str]): Doc/Token attributes assigned by this component,
|
|||
|
|
e.g. "token.ent_id". Used for pipeline analysis.
|
|||
|
|
requires (Iterable[str]): Doc/Token attributes required by this component,
|
|||
|
|
e.g. "token.ent_id". Used for pipeline analysis.
|
|||
|
|
retokenizes (bool): Whether the component changes the tokenization.
|
|||
|
|
Used for pipeline analysis.
|
|||
|
|
default_score_weights (Dict[str, Optional[float]]): The scores to report during
|
|||
|
|
training, and their default weight towards the final score used to
|
|||
|
|
select the best model. Weights should sum to 1.0 per component and
|
|||
|
|
will be combined and normalized for the whole pipeline. If None,
|
|||
|
|
the score won't be shown in the logs or be weighted.
|
|||
|
|
func (Optional[Callable]): Factory function if not used as a decorator.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#factory
|
|||
|
|
rØ ©Ú decoratorr r zdefault config©Ústyler‘ Úcfg_typeÚ factory_funcrl c
|
|||
|
|
óò •— ‰j ‰
|
|||
|
|
« }|t j v rbt j j |« }t j
|
|||
|
|
| |« s-t j j ‰
|
|||
|
|
|| ¬« }t |« ‚t j | « }d|vsd|vr)t t j j ‰
|
|||
|
|
¬« « ‚t j j || ¬« t ‰
|
|||
|
|
‰t ‰« t ‰ « t ‰ j! « « ‰ ‰ ¬« }‰j# ‰
|
|||
|
|
|« t% t j j' « t j( ¬« ‰_ | S )N)r‘ r Únew_funcro r‘ r )r )rØ r˜ r r ÚscoresrÝ r r† )r rG rð r¤ r$ Ú is_same_funcr& ÚE004r¢ r Ú
|
|||
|
|
get_arg_namesÚE964Úregisterrˆ r1 rÐ rÄ r rB r• ÚE957)
|
|||
|
|
r r Ú
|
|||
|
|
existing_funcÚerrÚ arg_namesÚ factory_metar r² r˜ rÝ r‘ r r s
|
|||
|
|
€€€€€€€rk Ú add_factoryz%Language.factory.<locals>.add_factoryý sD ø€ Ø × 0Ñ 0°Ó 6ˆMØ ¤× 2Ñ 2Ñ2ô
|
|||
|
|
!)× 2Ñ 2× 6Ñ 6°}Ó E�
|
|||
|
|
Ü×(Ñ(¨ °}ÔEÜ Ÿ+™+×,Ñ,Ø !¨
|
|||
|
|
À ð -ó �Cô % S›/Ð)ä ×*Ñ*¨<Ó8ˆIؘIÑ%¨°yÑ)@Ü ¤§¡×!3Ñ!3¸Ð!3Ó!>Ó?Ð?ô
|
|||
|
|
|
|||
|
|
× Ñ × 'Ñ '¨
|
|||
|
|
¸LÐ 'Ô IÜ&Ø Ø -Ü& wÓ/Ü'¨Ó1ÜÐ 1× 6Ñ 6Ó 8Ó9Ø&;Ø 'ô ˆLð
|
|||
|
|
× Ñ |Ô 4ô
|
|||
|
|
-Ü×"Ñ"×*Ñ*Ó,´F·K±Kô ˆCŒMð Ð rj ) rŸ rd r r& ÚE963r¢ ÚE853re ÚE962r£ r )
|
|||
|
|
r² r‘ r˜ r r r rÝ r r* r- s
|
|||
|
|
``````` rk rØ zLanguage.factoryÐ s¾ þ€ ôF ˜$¤Ô$Ü œVŸ[™[× /Ñ /¸)Ð /Ó DÓEÐ EØ �$‰;Ü œVŸ[™[× /Ñ /°TÐ /Ó :Ó;Ð ;ܘ.¬$Ô/Ü—+‘+×$Ñ$Ø&¨T¼DÀ Ó<Pð %ó ˆCô ˜S“/Ð !ð' ¤hð ' ´8÷ ' ó ' ðR Ð Ù ˜tÓ$Ð $ØÐrj ©r r r r .c óZ ‡ ‡‡‡‡‡‡— ‰�ft ‰t « s)t t j j d¬« « ‚d‰v r)t t j j ‰¬« « ‚‰�‰nt j ‰« Šdt dt fˆˆ ˆˆˆˆˆfd„ }‰� |‰« S |S )aØ Register a new pipeline component. Can be used for stateless function
|
|||
|
|
components that don't require a separate factory. Can be used as a
|
|||
|
|
decorator on a function or classmethod, or called as a function with the
|
|||
|
|
factory provided as the func keyword argument. To create a component and
|
|||
|
|
add it to the pipeline, you can use nlp.add_pipe(name).
|
|||
|
|
|
|||
|
|
name (str): The name of the component factory.
|
|||
|
|
assigns (Iterable[str]): Doc/Token attributes assigned by this component,
|
|||
|
|
e.g. "token.ent_id". Used for pipeline analysis.
|
|||
|
|
requires (Iterable[str]): Doc/Token attributes required by this component,
|
|||
|
|
e.g. "token.ent_id". Used for pipeline analysis.
|
|||
|
|
retokenizes (bool): Whether the component changes the tokenization.
|
|||
|
|
Used for pipeline analysis.
|
|||
|
|
func (Optional[Callable[[Doc], Doc]): Factory function if not used as a decorator.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#component
|
|||
|
|
r r r r Úcomponent_funcrl c óä •‡ — t ‰
|
|||
|
|
t « r)t t j j ‰ ¬« « ‚dt dt fˆ fd„ }‰j ‰ « }|t j v rdt j j |« }|j }|r |D �cg c] }|j ‘Œ c}d nd }t j |‰ « r|}‰j! ‰ ‰‰ ‰
|
|||
|
|
|¬« ‰ S c c}w )Nr r‘ rl c ó •— ‰S rµ ri )ro r‘ r3 s €rk r z?Language.component.<locals>.add_component.<locals>.factory_funcP s ø€ Ø%Ð%rj r r1 )rŸ r£ r r& ÚE965r¢ rd Ú PipeCallabler rG rð r¤ Ú __closure__Ú
|
|||
|
|
cell_contentsr$ r# rØ )r3 r r r) ÚclosureÚcÚwrappedr r² Úcomponent_namer r‘ r r s ` €€€€€€€rk Ú
|
|||
|
|
add_componentz)Language.component.<locals>.add_componentL sä ù€ ܘ$¤Ô%Ü ¤§¡×!3Ñ!3¸ Ð!3Ó!HÓIÐIð
|
|||
|
|
&¬ð
|
|||
|
|
&´ õ
|
|||
|
|
&ð × 0Ñ 0°Ó 6ˆMØ ¤× 2Ñ 2Ñ2ô !)× 2Ñ 2× 6Ñ 6°}Ó E�
|
|||
|
|
Ø'×3Ñ3�ÙCJ±GÓ<±G¨q˜1Ÿ?›?°GÑ<¸QÒ?ÐPT�Ü×$Ñ$ W¨nÔ=Ø#0�Là �K‰KØ Ø Ø!Ø 'Ø!ð
|
|||
|
|
ô
|
|||
|
|
ð "Ð !ùò =s Â"C-) rŸ rd r r& r. r¢ r/ r$ Úget_object_namer7 r )r² r‘ r r r r r> r= s `````` @rk r zLanguage.component* sž þ€ ð6 Ð Ü ˜d¤CÔ(Ü ¤§¡×!3Ñ!3¸kÐ!3Ó!JÓKÐKØ�d‰{Ü ¤§¡×!3Ñ!3¸Ð!3Ó!>Ó?Ð?Ø!%Ð!1™´t×7KÑ7KÈDÓ7Qˆð "¬,ð "¼8÷ "ó "ð> Ð Ù Ó&Ð &Ø Ð rj )r r r" r )rÄ ÚprettyrÄ r@ c ó> — t | |¬« }|r
|
|||
|
|
t ||¬« |S )a‚ Analyze the current pipeline components, print a summary of what
|
|||
|
|
they assign or require and check that all requirements are met.
|
|||
|
|
|
|||
|
|
keys (List[str]): The meta values to display in the table. Corresponds
|
|||
|
|
to values in FactoryMeta, defined by @Language.factory decorator.
|
|||
|
|
pretty (bool): Pretty-print the results.
|
|||
|
|
RETURNS (dict): The data.
|
|||
|
|
)rÄ )r/ r0 )rª rÄ r@ Úanalysiss rk r/ zLanguage.analyze_pipeso s" € ô ! ¨DÔ1ˆÙ Ü ¨tÕ 4؈rj c ó¤ — | j D ] \ }}||k( sŒ |c S t t j j || j
|
|||
|
|
¬« « ‚)zßGet a pipeline component for a given component name.
|
|||
|
|
|
|||
|
|
name (str): Name of pipeline component to get.
|
|||
|
|
RETURNS (callable): The pipeline component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#get_pipe
|
|||
|
|
©r‘ Úopts)r§ ÚKeyErrorr& ÚE001r¢ rÒ )rª r‘ râ r s rk Úget_pipezLanguage.get_pipe‚ sN € ð %)×$4Ô$4Ñ ˆI�yؘDÓ Ø Ò ð %5ô ”v—{‘{×)Ñ)¨t¸$×:NÑ:NÐ)ÓOÓPÐPrj )rQ Ú
|
|||
|
|
raw_configÚvalidateÚ factory_namerQ rI rJ c ó2 — |�|n|}t |t « s6t j j d|t |« ¬« }t
|
|||
|
|
|« ‚t j |« s)t
|
|||
|
|
t j j |¬« « ‚| j |« set j j |dj | j « dt j | « | j ¬« }t
|
|||
|
|
|« ‚| j# |« }|j$ r$t' |j$ « j) |« }| j+ |« }|t, j. vr|}| |dœ|¥d |i¥}||i} t- j0 | |¬
|
|||
|
|
« }
|
|||
|
|
t- j2 d | | i|¬
|
|||
|
|
« d } t' | « } || d < | j5 d d« | j5 d
|
|||
|
|
d« | j5 dd« |r| j) |« } | | j6 |< |
|
|||
|
|
| S )a Create a pipeline component. Mostly used internally. To create and
|
|||
|
|
add a component to the pipeline, you can use nlp.add_pipe.
|
|||
|
|
|
|||
|
|
factory_name (str): Name of component factory.
|
|||
|
|
name (Optional[str]): Optional name to assign to component instance.
|
|||
|
|
Defaults to factory name if not set.
|
|||
|
|
config (Dict[str, Any]): Config parameters to use for this component.
|
|||
|
|
Will be merged with default config, if available.
|
|||
|
|
raw_config (Optional[Config]): Internals: the non-interpolated config.
|
|||
|
|
validate (bool): Whether to validate the component config against the
|
|||
|
|
arguments and types expected by the factory.
|
|||
|
|
RETURNS (Callable[[Doc], Doc]): The pipeline component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#create_pipe
|
|||
|
|
NrQ r rÚ ú, Ú create_pipe)r‘ rE Úmethodr Ú lang_code)ro r‘ z
|
|||
|
|
@factories)rJ ÚcfgrØ ro r‘ ) rŸ re r& r0 r¢ r£ r rß rà rá r ÚE002Újoinrò r$ r? r r r˜ r r— r rG rð r¥ ÚfillÚpoprž ) rª rK r‘ rQ rI rJ r* rã r rQ ÚresolvedÚfilleds rk rN zLanguage.create_pipe� sí € ð0 Ð'‰t¨\ˆÜ˜&¤$Ô'Ü—+‘+×$Ñ$¨8¸$ÌÈfË Ð$ÓVˆCÜ ˜S“/Ð !Ü×)Ñ)¨&Ô1Ü œVŸ[™[× /Ñ /°vÐ /Ó >Ó?Ð ?Ø× Ñ Ô-Ü—+‘+×$Ñ$Ø!Ø—Y‘Y˜t× 1Ñ 1Ó2Ø$Ü×)Ñ)¨$Ó/Ø Ÿ)™)ð %ó ˆCô ˜S“/Ð !Ø×)Ñ)¨,Ó7ˆ ð × #Ò #ܘI× 4Ñ 4Ó5×;Ñ;¸FÓCˆFØ ×-Ñ-¨lÓ;ˆ
|
|||
|
|
ð ¤× 2Ñ 2Ñ 2Ø (ˆMð tÑS¨vÐS°|À]ÑSˆð ˜VÐ$ˆô ×#Ñ# C°(Ô;ˆÜ— ‘ s¨<Ñ'8Ð 9ÀHÔMÈeÑTˆÜ˜“ ˆØ (ˆˆyÑØ�
|
|||
|
|
‰
|
|||
|
|
�< Ô&ð �
|
|||
|
|
‰
|
|||
|
|
�5˜$Ô Ø�
|
|||
|
|
‰
|
|||
|
|
�6˜4Ô ñ Ø—\‘\ *Ó-ˆFØ#)ˆ×ј4Ñ Ø˜ Ñ%Ð%rj Ú source_nameÚsourcec óâ — t |t « s3t t j j |t
|
|||
|
|
|« ¬« « ‚| j j |j j k7 r3t j t j j |¬« « ||j vrct t j j ||j d › d|j d › �dj# |j « ¬« « ‚|j% |« }t' |d« r||_ |j* j- « }t/ j0 |d | « }|| j2 |< | j j4 |j j4 k7 r@|j j4 D ]' }| j j4 j7 |« Œ) ||d fS )
|
|||
|
|
au Create a pipeline component by copying it from an existing model.
|
|||
|
|
|
|||
|
|
source_name (str): Name of the component in the source pipeline.
|
|||
|
|
source (Language): The source nlp object to copy from.
|
|||
|
|
name (str): Optional alternative name to use in current pipeline.
|
|||
|
|
RETURNS (Tuple[Callable[[Doc], Doc], str]): The component and its factory name.
|
|||
|
|
)r‘ rY r r rí r‘ rM )r‘ ÚmodelrE rÇ rØ ) rŸ rm r r& ÚE945r¢ r£ r{ r� ÚwarningsÚwarnr' ÚW113rÒ rF ÚE944r‹ rS rH r r‘ rQ Ú interpolater$ Ú copy_configrž ÚstringsÚadd)rª rX rY r‘ rý Ú
|
|||
|
|
source_configrä Úss rk Úcreate_pipe_from_sourcez Language.create_pipe_from_sourceØ s� € ô ˜&¤(Ô+Ü œVŸ[™[× /Ñ /°[ÌÈfË Ð /Ó VÓWÐ WØ �:‰:× Ñ § ¡ ×!5Ñ!5Ò 5Ü �M‰Mœ(Ÿ-™-×.Ñ.°KÐ.Ó@Ô AØ ˜f× 4Ñ 4Ñ 4ÜÜ— ‘ ×"Ñ"Ø$Ø #Ÿ[™[¨Ñ 0Ð 1°°6·;±;¸vÑ3FÐ2GÐHØ Ÿ™ 6×#9Ñ#9Ó:ð #ó óð
|
|||
|
|
ð �‰˜{Ó+ˆô �4˜Ô Ø ˆDŒIð Ÿ
|
|||
|
|
™
|
|||
|
|
×1Ñ1Ó3ˆ
|
|||
|
|
Ü×&Ñ& }°\Ñ'BÀ;Ñ'OÓPˆ Ø#.ˆ×ј4Ñ Ø �:‰:× Ñ § ¡ ×!5Ñ!5Ò 5Ø—\‘\×)Ô)�Ø—
|
|||
|
|
‘
|
|||
|
|
×"Ñ"×&Ñ& qÕ)ð *à�[ Ñ+Ð+Ð+rj )ÚbeforeÚafterÚfirstÚlastrY rQ rI rJ rh ri rj rk c óÐ — t |t « s7t |« } t j j | |¬« } t
|
|||
|
|
| « ‚|�|n|}|| j v r4t
|
|||
|
|
t j j || j ¬« « ‚d|v rBt j t j j |j d« ¬« « |�| j |||¬« \ }
|
|||
|
|
}n| j |||| |
|
|||
|
|
¬« }
|
|||
|
|
| j! ||||« }| j# |« | j$ |< | j& j) |||
|
|||
|
|
f« | j+ « |
|
|||
|
|
S )aû Add a component to the processing pipeline. Valid components are
|
|||
|
|
callables that take a `Doc` object, modify it and return it. Only one
|
|||
|
|
of before/after/first/last can be set. Default behaviour is "last".
|
|||
|
|
|
|||
|
|
factory_name (str): Name of the component factory.
|
|||
|
|
name (str): Name of pipeline component. Overwrites existing
|
|||
|
|
component.name attribute if available. If no name is set and
|
|||
|
|
the component exposes no name attribute, component.__name__ is
|
|||
|
|
used. An error is raised if a name already exists in the pipeline.
|
|||
|
|
before (Union[str, int]): Name or index of the component to insert new
|
|||
|
|
component directly before.
|
|||
|
|
after (Union[str, int]): Name or index of the component to insert new
|
|||
|
|
component directly after.
|
|||
|
|
first (bool): If True, insert component first in the pipeline.
|
|||
|
|
last (bool): If True, insert component last in the pipeline.
|
|||
|
|
source (Language): Optional loaded nlp object to copy the pipeline
|
|||
|
|
component from.
|
|||
|
|
config (Dict[str, Any]): Config parameters to use for this component.
|
|||
|
|
Will be merged with default config, if available.
|
|||
|
|
raw_config (Optional[Config]): Internals: the non-interpolated config.
|
|||
|
|
validate (bool): Whether to validate the component config against the
|
|||
|
|
arguments and types expected by the factory.
|
|||
|
|
RETURNS (Callable[[Doc], Doc]): The pipeline component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#add_pipe
|
|||
|
|
©r r‘ rD r‘ )Úname_in_configr )r‘ rQ rI rJ )rŸ rd Úreprr& ÚE966r¢ r rÒ ÚE007r] r^ r' ÚW119rU rg rN Ú_get_pipe_indexr r� r§ ÚinsertÚ_link_components)rª rK r‘ rh ri rj rk rY rQ rI rJ Úbad_valr* Úpipe_componentÚ
|
|||
|
|
pipe_indexs rk Úadd_pipezLanguage.add_pipeÿ sX € ôP ˜,¬Ô,ܘ<Ó(ˆGÜ—+‘+×$Ñ$¨w¸TÐ$ÓBˆCÜ ˜S“/Ð !ØÐ'‰t¨\ˆØ �4×'Ñ'Ñ 'Ü œVŸ[™[× /Ñ /°TÀ×@TÑ@TÐ /Ó UÓVÐ Và �VÑ Ü �M‰Mœ(Ÿ-™-×.Ñ.¸f¿j¹jÈÓ>PÐ.ÓQÔ RØ Ð ð ,0×+GÑ+GØ ˜f¨4ð ,Hó ,Ñ (ˆN™Lð "× -Ñ -Ø ØØ Ø%Ø!ð .ó ˆNð ×)Ñ)¨&°%¸ÀÓEˆ
|
|||
|
|
Ø $× 5Ñ 5°lÓ Cˆ�‰˜Ñ Ø ×Ñ× Ñ
|
|||
|
|
¨T°>Ð,BÔCØ × Ñ Ô Ø Ð rj c óÐ — ||||dœ}t d„ ||||fD « « dk\ r4t t j j || j
|
|||
|
|
¬« « ‚|st
|
|||
|
|
d„ |||fD « « st | j « S |ryt |t « r]|| j
|
|||
|
|
vr4t t j j || j
|
|||
|
|
¬« « ‚| j
|
|||
|
|
j |« S t |t « r`|| j
|
|||
|
|
vr4t t j j || j
|
|||
|
|
¬« « ‚| j
|
|||
|
|
j |« dz S t |« t k( rV|t | j « k\ s|dk r7t j j d || j
|
|||
|
|
¬
|
|||
|
|
« }t |« ‚|S t |« t k( rY|t | j « k\ s|dk r7t j j d || j
|
|||
|
|
¬
|
|||
|
|
« }t |« ‚|dz S t t j j || j
|
|||
|
|
¬« « ‚) aÔ Determine where to insert a pipeline component based on the before/
|
|||
|
|
after/first/last values.
|
|||
|
|
|
|||
|
|
before (str): Name or index of the component to insert directly before.
|
|||
|
|
after (str): Name or index of component to insert directly after.
|
|||
|
|
first (bool): If True, insert component first in the pipeline.
|
|||
|
|
last (bool): If True, insert component last in the pipeline.
|
|||
|
|
RETURNS (int): The index of the new pipeline component.
|
|||
|
|
)rh ri rj rk c 3 ó$ K — | ] }|d u–— Œ
|
|||
|
|
y wrµ ri )Ú.0Úargs rk ú <genexpr>z+Language._get_pipe_index.<locals>.<genexpr>V s è ø€ ÐGÑ*F 3ˆs˜$ŒÑ*Fùó ‚é )ÚargsrE c 3 ó$ K — | ] }|d u–— Œ
|
|||
|
|
y wrµ ri )r| rÓ s rk r~ z+Language._get_pipe_index.<locals>.<genexpr>Z s è ø€ ÐQÑ:P°˜5¨Ô,Ñ:Pùr r rD r! rh )ÚdirÚidxrE ri )Úsumr r& ÚE006r¢ rÒ ÚanyrÍ r§ rŸ rd rG Úindexr£ rf ÚE959)rª rh ri rj rk Úall_argsr* s rk rs zLanguage._get_pipe_indexE s' € ð %¨u¸uÈdÑSˆÜ ÑG¨6°5¸%ÀÑ*FÓGÓ GÈ1Ò LÜ Ü— ‘ ×"Ñ"¨°t×7KÑ7KÐ"ÓLóð
|
|||
|
|
ñ ”sÑQ¸5À&È%Ñ:PÓQÔQÜ�t×'Ñ'Ó(Ð (Ù
|
|||
|
|
ØÜ
|
|||
|
|
˜¤Ô
|
|||
|
|
$ؘT× 1Ñ 1Ñ1Ü Ü—K‘K×&Ñ&¨F¸×9MÑ9MÐ&ÓNóð ð ×'Ñ'×-Ñ-¨fÓ5Ð 5Ü
|
|||
|
|
˜œsÔ
|
|||
|
|
#ؘD× 0Ñ 0Ñ0Ü Ü—K‘K×&Ñ&¨E¸×8LÑ8LÐ&ÓMóð ð ×'Ñ'×-Ñ-¨eÓ4°qÑ8Ð 8ô �&‹\œSÒ
|
|||
|
|
Øœ˜T× -Ñ -Ó.Ò.°&¸1²*Ü —k‘k×(Ñ(Ø f°4×3GÑ3Gð )ó �ô ! “oÐ%؈MÜ
|
|||
|
|
�%‹[œCÒ
|
|||
|
|
Øœ˜D× ,Ñ ,Ó-Ò-°¸²Ü —k‘k×(Ñ(Ø U°×1EÑ1Eð )ó �ô ! “oÐ%ؘ1‘9Ð ÜœŸ™×+Ñ+°À×@TÑ@TÐ+ÓUÓVÐVrj c ó — || j v S )a Check if a component name is present in the pipeline. Equivalent to
|
|||
|
|
`name in nlp.pipe_names`.
|
|||
|
|
|
|||
|
|
name (str): Name of the component.
|
|||
|
|
RETURNS (bool): Whether a component of the name exists in the pipeline.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#has_pipe
|
|||
|
|
)rÑ r s rk Úhas_pipezLanguage.has_pipe| s € ð �t—‘Ð&Ð&rj )rQ rJ c ó — || j vr4t t j j || j
|
|||
|
|
¬« « ‚t
|
|||
|
|
|d« r5t j j t |« |¬« }t |« ‚| j j |« }| j |« t | j « r|t | j « k( r| j ||||¬« S | j |||||¬« S )aB Replace a component in the pipeline.
|
|||
|
|
|
|||
|
|
name (str): Name of the component to replace.
|
|||
|
|
factory_name (str): Factory name of replacement component.
|
|||
|
|
config (Optional[Dict[str, Any]]): Config parameters to use for this
|
|||
|
|
component. Will be merged with default config, if available.
|
|||
|
|
validate (bool): Whether to validate the component config against the
|
|||
|
|
arguments and types expected by the factory.
|
|||
|
|
RETURNS (Callable[[Doc], Doc]): The new pipeline component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#replace_pipe
|
|||
|
|
rD Ú__call__rm )r‘ rQ rJ )r‘ rh rQ rJ )rÒ r r& rG r¢ rÑ r ÚE968ro rˆ Ú remove_piperÍ r§ ry )rª r‘ rK rQ rJ r* rx s rk Ú replace_pipezLanguage.replace_pipe‡ só € ð( �t×+Ñ+Ñ +Ü œVŸ[™[× /Ñ /°TÀÇÁÐ /Ó PÓQÐ QÜ �< Ô ,Ü—+‘+×$Ñ$¬t°LÓ/AÈÐ$ÓMˆCÜ ˜S“/Ð !ð ×)Ñ)×/Ñ/°Ó5ˆ
|
|||
|
|
Ø ×Ñ˜Ô Ü�4×#Ñ#Ô$¨
|
|||
|
|
´c¸$×:JÑ:JÓ6KÒ(Kà—=‘=Ø 4°Àð !ó ð
|
|||
|
|
ð —=‘=Ø ØØ!Ø Ø!ð !ó ð
|
|||
|
|
rj Úold_nameÚnew_namec óØ — || j vr4t t j j || j ¬« « ‚|| j v r4t t j
|
|||
|
|
j || j ¬« « ‚| j j
|
|||
|
|
|« }|| j | d f| j |< | j j |« | j |< | j j |« | j |< || j d d v r6| j d d j |« }|| j d d |< | j « y)zËRename a pipeline component.
|
|||
|
|
|
|||
|
|
old_name (str): Name of the component to rename.
|
|||
|
|
new_name (str): New name of the component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#rename_pipe
|
|||
|
|
rD r! Ú
|
|||
|
|
initializerÇ N)
|
|||
|
|
rÒ r r& rG r¢ rq rˆ r§ r� rU rž r™ ru )rª r’ r“ ÚiÚinit_cfgs rk Ú rename_pipezLanguage.rename_pipe² sI € ð ˜4×/Ñ/Ñ /Ü Ü— ‘ ×"Ñ"¨°t×7KÑ7KÐ"ÓLóð
|
|||
|
|
ð �t×+Ñ+Ñ +Ü Ü— ‘ ×"Ñ"¨°t×7KÑ7KÐ"ÓLóð
|
|||
|
|
ð
|
|||
|
|
× Ñ × &Ñ & xÓ 0ˆØ '¨×)9Ñ)9¸!Ñ)<¸QÑ)?Ð @ˆ×јÑØ$(§O¡O×$7Ñ$7¸Ó$Aˆ�‰˜Ñ!Ø'+×'9Ñ'9×'=Ñ'=¸hÓ'Gˆ×ј8Ñ$à �t—|‘| LÑ1°,Ñ?Ñ ?Ø—|‘| LÑ1°,Ñ?×CÑCÀHÓMˆHØAIˆD�L‰L˜ Ñ & |Ñ 4°XÑ >Ø × Ñ Õ rj c ó’ — || j vr4t t j j || j ¬« « ‚| j
|
|||
|
|
j
|
|||
|
|
| j j |« « }| j j
|
|||
|
|
|« | j j
|
|||
|
|
|« | j j di « j
|
|||
|
|
|d« || j d d v r!| j d d j
|
|||
|
|
|« || j v r| j j |« | j! « |S )a Remove a component from the pipeline.
|
|||
|
|
|
|||
|
|
name (str): Name of the component to remove.
|
|||
|
|
RETURNS (Tuple[str, Callable[[Doc], Doc]]): A `(name, component)` tuple of the removed component.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#remove_pipe
|
|||
|
|
rD Ú_sourced_vectors_hashesNr• rÇ )rÒ r r& rG r¢ r§ rU rˆ r� rž r‹ r¤ r™ rÈ r¨ Úremoveru )rª r‘ Úremoveds rk r� zLanguage.remove_pipeÌ s € ð �t×+Ñ+Ñ +Ü œVŸ[™[× /Ñ /°TÀ×@TÑ@TÐ /Ó UÓVÐ VØ×"Ñ"×&Ñ& t×';Ñ';×'AÑ'AÀ$Ó'GÓHˆð
|
|||
|
|
�‰×јDÔ!Ø ×Ñ× Ñ ˜tÔ$Ø � ‰ �
|
|||
|
|
‰
|
|||
|
|
Ð/°Ó4×8Ñ8¸¸tÔDà �4—<‘< Ñ-¨lÑ;Ñ ;Ø �L‰L˜ Ñ & |Ñ 4× 8Ñ 8¸Ô >à �4—=‘=Ñ Ø �N‰N× !Ñ ! $Ô 'Ø × Ñ Ô Øˆrj c ó¾ — || j vr4t t j j || j ¬« « ‚| j
|
|||
|
|
j
|
|||
|
|
|« y)a Disable a pipeline component. The component will still exist on
|
|||
|
|
the nlp object, but it won't be run as part of the pipeline. Does
|
|||
|
|
nothing if the component is already disabled.
|
|||
|
|
|
|||
|
|
name (str): The name of the component to disable.
|
|||
|
|
rD N)rÒ r r& rG r¢ r¨ rd r s rk Ú disable_pipezLanguage.disable_pipeå sI € ð �t×+Ñ+Ñ +Ü œVŸ[™[× /Ñ /°TÀ×@TÑ@TÐ /Ó UÓVÐ VØ �‰×ј4Õ rj c óÜ — || j vr4t t j j || j ¬« « ‚|| j
|
|||
|
|
v r | j j |« yy)zÑEnable a previously disabled pipeline component so it's run as part
|
|||
|
|
of the pipeline. Does nothing if the component is already enabled.
|
|||
|
|
|
|||
|
|
name (str): The name of the component to enable.
|
|||
|
|
rD N)rÒ r r& rG r¢ rÈ r¨ r› r s rk Ú enable_pipezLanguage.enable_pipeð sZ € ð �t×+Ñ+Ñ +Ü œVŸ[™[× /Ñ /°TÀ×@TÑ@TÐ /Ó UÓVÐ VØ �4—=‘=Ñ Ø �N‰N× !Ñ ! $Õ 'ð !rj )ÚdisableÚ
|
|||
|
|
component_cfgÚtextr¡ r¢ c óš — | j |« }|€i }| j D ]Î \ }}||v rŒ t |d« s3t t j
|
|||
|
|
j
|
|||
|
|
t |« |¬« « ‚| j }t |d« r|j « } ||fi |j |i « ¤Ž}t |t « rŒ�t t j j
|
|||
|
|
|t |« ¬« « ‚ |S # t $ r/}t t j j
|
|||
|
|
|¬« « |‚d}~wt $ r} ||||g|« Y d}~Œ—d}~ww xY w)a¬ Apply the pipeline to some text. The text can span multiple sentences,
|
|||
|
|
and can contain arbitrary whitespace. Alignment into the original string
|
|||
|
|
is preserved.
|
|||
|
|
|
|||
|
|
text (Union[str, Doc]): If `str`, the text to be processed. If `Doc`,
|
|||
|
|
the doc will be passed directly to the pipeline, skipping
|
|||
|
|
`Language.make_doc`.
|
|||
|
|
disable (List[str]): Names of the pipeline components to disable.
|
|||
|
|
component_cfg (Dict[str, dict]): An optional dictionary with extra
|
|||
|
|
keyword arguments for specific components.
|
|||
|
|
RETURNS (Doc): A container for accessing the annotations.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#call
|
|||
|
|
NrŽ rm Úget_error_handlerr )r‘ Ú
|
|||
|
|
returned_type)Ú _ensure_docr¹ r r r& ÚE003r¢ r£ r© r¥ r¤ rF ÚE109Ú ExceptionrŸ r: ÚE005) rª r£ r¡ r¢ Údocr‘ ÚprocÚ
|
|||
|
|
error_handlerÚes rk rŽ zLanguage.__call__û s2 € ð* × Ñ ˜tÓ$ˆØ Ð Ø ˆMØ Ÿ-œ-‰JˆD�$Ø�w‰Øܘ4 Ô,Ü ¤§¡×!3Ñ!3¼dÀ4»jÈtÐ!3Ó!TÓUÐUØ × 6Ñ 6ˆMÜ�tÐ 0Ô1Ø $× 6Ñ 6Ó 8�
|
|||
|
|
ð
|
|||
|
|
4Ù˜3Ñ> -×"3Ñ"3°D¸"Ó"=Ñ>�ô ˜c¤3Õ'Ü ¤§¡×!3Ñ!3¸ÌTÐRUËYÐ!3Ó!WÓXÐXð! (ð" ˆ
|
|||
|
|
øô ò
|
|||
|
|
Gä ¤§¡×!3Ñ!3¸Ð!3Ó!>Ó?ÀQÐFûÜ ò
|
|||
|
|
4Ù ˜d D¨3¨%°×3Ñ3ûð
|
|||
|
|
4ús$ ÂC5Ã5 E
|
|||
|
|
Ã>*D(Ä( E
|
|||
|
|
Ä4 EÅE
|
|||
|
|
Ú
|
|||
|
|
DisabledPipesc óÎ — t j t j t « t |« dk( r t
|
|||
|
|
|d t t f« r|d }| j |¬« S )aV Disable one or more pipeline components. If used as a context
|
|||
|
|
manager, the pipeline will be restored to the initial state at the end
|
|||
|
|
of the block. Otherwise, a DisabledPipes object is returned, that has
|
|||
|
|
a `.restore()` method you can use to undo your changes.
|
|||
|
|
|
|||
|
|
This method has been deprecated since 3.0
|
|||
|
|
r! r )r¡ )
|
|||
|
|
r] r^ r' ÚW096ÚDeprecationWarningrÍ rŸ rÐ ÚtupleÚ select_pipesrñ s rk Ú
|
|||
|
|
disable_pipeszLanguage.disable_pipes& sP € ô �
|
|||
|
|
‰
|
|||
|
|
”h—m‘mÔ%7Ô8Ü ˆu‹:˜Š?œz¨%°©(´T¼5°MÔ Bؘ!‘HˆEØ× Ñ ¨Ð Ó/Ð/rj )r¡ Úenabler· c ó´ — |€|€t t j « ‚t |t « r|g}|�ot |t « r|g}| j
|
|||
|
|
D �cg c] }||vsŒ|‘Œ }}|�:||k7 r5t t j j ||| j
|
|||
|
|
¬« « ‚|}|€J ‚|D �cg c] }|| j vsŒ|‘Œ }}t | |« S c c}w c c}w )a Disable one or more pipeline components. If used as a context
|
|||
|
|
manager, the pipeline will be restored to the initial state at the end
|
|||
|
|
of the block. Otherwise, a DisabledPipes object is returned, that has
|
|||
|
|
a `.restore()` method you can use to undo your changes.
|
|||
|
|
|
|||
|
|
disable (str or iterable): The name(s) of the pipes to disable
|
|||
|
|
enable (str or iterable): The name(s) of the pipes to enable - all others will be disabled
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#select_pipes
|
|||
|
|
)r· r¡ rî )
|
|||
|
|
r r& ÚE991rŸ rd rÑ ÚE992r¢ r¨ r° )rª r¡ r· rý Ú
|
|||
|
|
to_disableÚds rk rµ zLanguage.select_pipes3 sì € ð ˆ>˜g˜oÜ œVŸ[™[Ó)Ð )Ü �gœsÔ #Ø �iˆGØ Ð Ü˜&¤#Ô&Ø ˜�Ø+/¯?ª?ÓQ©? 4¸dÈ&Ò>Pš$¨?ˆJÐQàÐ" w°*Ò'<Ü Ü—K‘K×&Ñ&Ø %¨w¸d¿o¹oð 'ó óð ð
|
|||
|
|
!ˆGØÐ"Ð"Ð"ñ &ÓA™g˜¨°$·.±.Ò)@’1˜gˆÐAÜ ˜T 7Ó+Ð+ùò Rùò Bs Á CÁ CÂ*CÂ>Cc óÎ — t |« | j kD r=t t j j t |« | j ¬« « ‚| j
|
|||
|
|
|« S )z{Turn a text into a Doc object.
|
|||
|
|
|
|||
|
|
text (str): The text to process.
|
|||
|
|
RETURNS (Doc): The processed doc.
|
|||
|
|
)ÚlengthrŠ )rÍ rŠ r r& ÚE088r¢ r“ )rª r£ s rk Úmake_doczLanguage.make_docY sQ € ô ˆt‹9�t—‘Ò &Ü Ü— ‘ ×"Ñ"¬#¨d«)ÀÇÁÐ"ÓPóð
|
|||
|
|
ð �~‰~˜dÓ#Ð#rj Údoc_likec ó4 — t |t « r|S t |t « r| j |« S t |t « r$t | j
|
|||
|
|
« j
|
|||
|
|
|« S t t j j t |« ¬« « ‚)z„Create a Doc if need be, or raise an error if the input is not
|
|||
|
|
a Doc, string, or a byte array (generated by Doc.to_bytes()).)r£ ) rŸ r: rd rÀ Úbytesr{ Ú
|
|||
|
|
from_bytesr r& ÚE1041r¢ r£ )rª rÁ s rk r§ zLanguage._ensure_doce sp € ô �h¤Ô $؈OÜ �h¤Ô $Ø—=‘= Ó*Ð *Ü �h¤Ô &Ü�t—z‘z“?×-Ñ-¨hÓ7Ð 7ÜœŸ ™ ×,Ñ,´$°x³.Ð,ÓAÓBÐBrj Úcontextc ó6 — | j |« }||_ |S )z>Call _ensure_doc to generate a Doc and set its context object.)r§ Ú_context)rª rÁ rÆ r¬ s rk Ú_ensure_doc_with_contextz!Language._ensure_doc_with_contextp s € ð × Ñ ˜xÓ(ˆØ ˆŒ ؈
|
|||
|
|
rj g )ÚdropÚsgdÚlossesr¢ ÚexcludeÚ annotatesÚexamplesrí rÊ rË rÌ rÍ rÎ c ó´ — |�t t j « ‚|€i }t |t « rt |« dk( r|S t
|
|||
|
|
|d« t |« }|€-| j €| j « | _ | j }|€i }i } t | j « D ]_ \ }
|
|||
|
|
\ } } |j | i « t || « | | < || j d|« | | j d| j « Œa | j D ]½ \ } } | |vr%t | d« r | j |fd|dœ|| ¤Ž |dvrI| |vrEt | t" j$ « r+| j& r | j( d vr| j+ |« | |v sŒ�t- t/ d
|
|||
|
|
„ |D « | | | j0 | | ¬ « |« D ] \ }
|
|||
|
|
}|
|
|||
|
|
|_ Œ Œ¿ t5 |« S ) a< Update the models in the pipeline.
|
|||
|
|
|
|||
|
|
examples (Iterable[Example]): A batch of examples
|
|||
|
|
_: Should not be set - serves to catch backwards-incompatible scripts.
|
|||
|
|
drop (float): The dropout rate.
|
|||
|
|
sgd (Optimizer): An optimizer.
|
|||
|
|
losses (Dict[str, float]): Dictionary to update with the loss, keyed by
|
|||
|
|
component.
|
|||
|
|
component_cfg (Dict[str, Dict]): Config parameters for specific pipeline
|
|||
|
|
components, keyed by component name.
|
|||
|
|
exclude (Iterable[str]): Names of components that shouldn't be updated.
|
|||
|
|
annotates (Iterable[str]): Names of components that should set
|
|||
|
|
annotations on the predicted examples after updating.
|
|||
|
|
RETURNS (Dict[str, float]): The updated losses dictionary
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#update
|
|||
|
|
Nr zLanguage.updaterÊ r� Úupdate©rË rÌ )NF)TFNc 3 ó4 K — | ] }|j –— Œ y wrµ ©Ú predicted©r| Úegs rk r~ z"Language.update.<locals>.<genexpr>¸ s è ø€ Ð9±¨"˜Ÿ � ±ùó ‚)r r‘ r© r« )r r& ÚE989rŸ rÐ rÍ r= Ú_copy_examplesrœ Úcreate_optimizerÚ enumerater¹ rË r r� r rÑ r# ÚTrainableComponentÚ is_trainabler[ Ú
|
|||
|
|
finish_updateÚziprD r© rÕ Ú_replace_numpy_floats)rª rÏ rí rÊ rË rÌ r¢ rÍ rÎ Ú pipe_kwargsr– r‘ r r¬ r× s rk rÑ zLanguage.updatex sÛ € ð:
|
|||
|
|
ˆ=Ü œVŸ[™[Ó)Ð )Ø ˆ>؈FÜ �h¤Ô %¬#¨h«-¸1Ò*<؈Mܘ(Ð$5Ô6Ü! (Ó+ˆØ ˆ;Ø�‰Ð&Ø"&×"7Ñ"7Ó"9�”Ø—/‘/ˆCØ Ð Ø ˆM؈ Ü (¨¯ © Ö 7‰OˆA‰|��dØ × $Ñ $ T¨2Ô .Ü (¨ °tÑ)<Ó =ˆK˜Ñ Ø ˜$Ñ × *Ñ *¨6°4Ô 8Ø ˜Ñ × (Ñ (¨ °t·±Õ Gð 8ð
|
|||
|
|
Ÿ-œ-‰JˆD�$à˜7Ñ"¤w¨t°XÔ'>Ø�— ‘ ˜HÐU¨$°vÑUÀ ÈtÑATÒUؘ-Ñ'à Ñ'Ü" 4¬×)>Ñ)>Ô?Ø ×)Ò)Ø Ÿ
|
|||
|
|
™
|
|||
|
|
Ð*=Ñ=à×&Ñ& sÔ+Ø�yÒ Ü "ÜÙ9±Ó9Ø !Ø !Ø.2×.HÑ.HØ *¨4Ñ 0ô ð ö ‘G�C˜ð $'�B•Lñ ð (ô0 % VÓ,Ð,rj )rË rÌ r¢ rÍ c óf ‡
|
|||
|
|
— |€i }t |t « rt |« dk( r|S t |d« |€-| j €| j « | _ | j }t | j « }t j |« |€i }i Š
|
|||
|
|
ˆ
|
|||
|
|
fd„}|j |_ |j |_
|
|||
|
|
|j |_ |D ]> \ }} ||v s t | d« sŒi Š
|
|||
|
|
| j |f||dœ|j |i « ¤Ž Œ@ ‰
|
|||
|
|
j « D ] \ }
|
|||
|
|
\ } } ||
|
|||
|
|
| | « Œ |S )a| Make a "rehearsal" update to the models in the pipeline, to prevent
|
|||
|
|
forgetting. Rehearsal updates run an initial copy of the model over some
|
|||
|
|
data, and update the model so its current predictions are more like the
|
|||
|
|
initial ones. This is useful for keeping a pretrained model on-track,
|
|||
|
|
even if you're updating it with a smaller set of examples.
|
|||
|
|
|
|||
|
|
examples (Iterable[Example]): A batch of `Example` objects.
|
|||
|
|
sgd (Optional[Optimizer]): An optimizer.
|
|||
|
|
component_cfg (Dict[str, Dict]): Config parameters for specific pipeline
|
|||
|
|
components, keyed by component name.
|
|||
|
|
exclude (Iterable[str]): Names of components that shouldn't be updated.
|
|||
|
|
RETURNS (dict): Results from the update.
|
|||
|
|
|
|||
|
|
EXAMPLE:
|
|||
|
|
>>> raw_text_batches = minibatch(raw_texts)
|
|||
|
|
>>> for labelled_batch in minibatch(examples):
|
|||
|
|
>>> nlp.update(labelled_batch)
|
|||
|
|
>>> raw_batch = [Example.from_dict(nlp.make_doc(text), {}) for text in next(raw_text_batches)]
|
|||
|
|
>>> nlp.rehearse(raw_batch)
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#rehearse
|
|||
|
|
r zLanguage.rehearsec ó •— ||f‰| < ||fS rµ ri )ÚkeyÚWÚdWÚgradss €rk Ú get_gradsz$Language.rehearse.<locals>.get_gradsñ s ø€ ؘR˜ˆE�#‰JØ�b�5ˆLrj ÚrehearserÒ )rŸ rÐ rÍ r= rœ rÛ r¹ ÚrandomÚshuffleÚ
|
|||
|
|
learn_rateÚb1Úb2r rê r¤ Úitems)rª rÏ rË rÌ r¢ rÍ rú ré r‘ r rå ræ rç rè s @rk rê zLanguage.rehearseà s0 ø€ ð> ˆ>؈FÜ �h¤Ô %¬#¨h«-¸1Ò*<؈Mܘ(Ð$7Ô8Ø ˆ;Ø�‰Ð&Ø"&×"7Ñ"7Ó"9�”Ø—/‘/ˆCÜ�T—]‘]Ó#ˆÜ�‰�uÔ Ø Ð Ø ˆM؈ô ð #Ÿ~™~ˆ Ô Ø—v‘vˆ Œ Ø—v‘vˆ Œ Û ‰JˆD�$Ø�w‰¤g¨d°JÔ&?Ø؈EØ ˆD�M‰MØð
|
|||
|
|
Ø '°ñ
|
|||
|
|
Ø:G×:KÑ:KÈDÐRTÓ:Uó
|
|||
|
|
ð ð "ŸK™KžM‰LˆC‘�!�RÙ ��Q˜�Oð *àˆ
|
|||
|
|
rj ©rË Ú get_examplesc óx — t j t j t « | j ||¬« S )Nrñ )r] r^ r' ÚW089r³ r• )rª rò rË s rk Úbegin_trainingzLanguage.begin_training s* € ô �
|
|||
|
|
‰
|
|||
|
|
”h—m‘mÔ%7Ô8Ø�‰˜|°ˆÓ5Ð5rj c óª ‡— |€=t j j d« t | j g d¢¬« Šˆfd„}t |d« s5t j j dt |« ¬« }t |« ‚| j j « }t j |d t ¬ « }|d
|
|||
|
|
}|� || « t! | |d |d |d
|
|||
|
|
¬« | j j& j( d dk\ r/t+ « }| j j& j- |« t | j. d« rEt1 | j. j2 |d dd¬« } | j. j2 |fd| i|¤Ž | j4 D ]c \ } }
|
|||
|
|
t7 |
|
|||
|
|
t8 j: « sŒ!|d j= | i « } t1 |
|
|||
|
|
j2 | d| ¬« } |
|
|||
|
|
j2 |fd| i| ¤Ž Œe |j= d« } | r(t j | t> ¬ « }
|
|||
|
|
tA | |
|
|||
|
|
|« | jC « || _" |�|| _" n!| jD €| jG « | _" |d }|� || « | jD S # t" $ r- t# t j$ j |d
|
|||
|
|
¬« « ‚w xY w)aÔ Initialize the pipe for training, using data examples if available.
|
|||
|
|
|
|||
|
|
get_examples (Callable[[], Iterable[Example]]): Optional function that
|
|||
|
|
returns gold-standard Example objects.
|
|||
|
|
sgd (Optional[Optimizer]): An optimizer to use for updates. If not
|
|||
|
|
provided, will be created using the .create_optimizer() method.
|
|||
|
|
RETURNS (thinc.api.Optimizer): The optimizer.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#initialize
|
|||
|
|
zUNo 'get_examples' callback provided to 'Language.initialize', creating dummy examples)ÚxÚyÚz)Úwordsc ó2 •— t j ‰ i « gS rµ )r< Ú from_dict)r¬ s €rk rò z)Language.initialize.<locals>.get_examples" s ø€ Ü ×)Ñ)¨#¨rÓ2Ð3Ð3rj rŽ zLanguage.initialize)rO Úobjr• )ÚschemaÚ before_initÚ
|
|||
|
|
vocab_datarƒ r� )Údatarƒ r� )r� r! r“ )Úsectionr‘ ro rÇ Ú pretrainingÚ
|
|||
|
|
after_init)$r$ r� r‚ r: r{ r r& ÚE930r¢ r£ Ú TypeErrorrQ ra rG r¥ r3 r? ÚIOErrorÚE884r� Úshaper Úto_opsr“ r6 r• r¹ rŸ r# ÚInitializableComponentr¤ r5 r> ru rœ rÛ )rª rò rË r* rQ ÚIrÿ ÚopsÚ tok_settingsr‘ r Ú
|
|||
|
|
p_settingsÚ pretrain_cfgÚPr r¬ s @rk r• zLanguage.initialize s’ ø€ ð Ð Ü �K‰K× Ñ Øgô
|
|||
|
|
ô �d—j‘jªÔ8ˆCô
|
|||
|
|
4ô �| ZÔ0Ü—+‘+×$Ñ$Ø,´$°|Ó2Dð %ó ˆCô ˜C“.Ð à—‘×(Ñ(Ó*ˆä × Ñ ˜V LÑ 1Ô:JÔ KˆØ˜
|
|||
|
|
Ñ&ˆ Ø Ð "Ù ˜Ô ð DÜ Ø˜1˜\™?°A°i±LÈ!ÈIÉ,õ
|
|||
|
|
ð
|
|||
|
|
�:‰:× Ñ × #Ñ # AÑ &¨!Ò +Ü!Ó#ˆCØ �J‰J× Ñ × %Ñ % cÔ *Ü �4—>‘> <Ô 0Ü1Ø—‘×)Ñ)Ø�+‘Ø#Ø ô ˆLð
|
|||
|
|
&ˆD�N‰N× %Ñ % lÑ M¸Ð MÀ Ò MØ Ÿ-œ-‰JˆD�$ܘ$¤× 9Ñ 9Õ:Ø ˜|™_× 0Ñ 0°°rÓ :�
|
|||
|
|
Ü 3Ø—O‘O Z¸ ÈDô �
|
|||
|
|
ð �—‘ ÑE°$ÐE¸*ÓEð
|
|||
|
|
(ð —z‘z -Ó0ˆ Ù Ü× Ñ Ô6JÔKˆAÜ ˜˜q !Ô $Ø × Ñ Ô Ø ˆŒØ ˆ?Ø !ˆD�OØ
|
|||
|
|
�_‰_Ð
|
|||
|
|
$Ø "× 3Ñ 3Ó 5ˆDŒOØ�|‘_ˆ
|
|||
|
|
Ø Ð !Ù �tÔ Ø�‰Ð øôC ò DÜœ&Ÿ+™+×,Ñ,°Q°y±\Ð,ÓBÓCÐ Cð Dús Ã
|
|||
|
|
J Ê 6Kc ó¦ — t « }| j j j d dk\ r%| j j j |« | j
|
|||
|
|
D ], \ }}t
|
|||
|
|
|d« sŒt |j « |_ Œ. |�|| _
|
|||
|
|
| j S | j €| j « | _
|
|||
|
|
| j S )a Continue training a pretrained model.
|
|||
|
|
|
|||
|
|
Create and return an optimizer, and initialize "rehearsal" for any pipeline
|
|||
|
|
component that has a .rehearse() method. Rehearsal is used to prevent
|
|||
|
|
models from "forgetting" their initialized "knowledge". To perform
|
|||
|
|
rehearsal, collect samples of text you want the models to retain performance
|
|||
|
|
on, and call nlp.rehearse() with a batch of Example objects.
|
|||
|
|
|
|||
|
|
RETURNS (Optimizer): The optimizer.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#resume_training
|
|||
|
|
r! Ú_rehearsal_model) r r{ r� r r
|
|||
|
|
r¹ r r r[ r rœ rÛ )rª rË r
|
|||
|
|
r‘ r s rk Úresume_trainingzLanguage.resume_trainingX s¯ € ô Ó ˆØ �:‰:× Ñ × #Ñ # AÑ &¨!Ò +Ø �J‰J× Ñ × %Ñ % cÔ *Ø Ÿ-œ-‰JˆD�$Ü�tÐ /Õ0Ü(0°·±Ó(<�Õ%ð (ð ˆ?Ø !ˆDŒOð �‰Ð ð �_‰_Ð
|
|||
|
|
$Ø "× 3Ñ 3Ó 5ˆDŒOØ�‰Ð rj r® c óv — || _ | j D ]# \ }}t |d« sŒ|j |« Œ% y)a Set an error handler object for all the components in the pipeline
|
|||
|
|
that implement a set_error_handler function.
|
|||
|
|
|
|||
|
|
error_handler (Callable[[str, Callable[[Doc], Doc], List[Doc], Exception], NoReturn]):
|
|||
|
|
Function that deals with a failing batch of documents. This callable
|
|||
|
|
function should take in the component's name, the component itself,
|
|||
|
|
the offending batch of documents, and the exception that was thrown.
|
|||
|
|
DOCS: https://spacy.io/api/language#set_error_handler
|
|||
|
|
Úset_error_handlerN)r© r¹ r r )rª r® r‘ rý s rk r zLanguage.set_error_handlerq s7 € ð &3ˆÔ"Ø Ÿ-œ-‰JˆD�$Ü�tÐ 0Õ1Ø×&Ñ& }Õ5ñ (rj )r� Úscorerr¢ Ú
|
|||
|
|
scorer_cfgÚ
|
|||
|
|
per_componentr r r c ó( — t |« }t |d« t |« }|€ | j }|€i }|€i }|€(t |« }|j d| « t
|
|||
|
|
di |¤Ž}t « }|D ]' } | j | j j « Œ) | j d„ |D « ||¬« }
|
|||
|
|
t ||
|
|||
|
|
« D ] \ } } | | _
|
|||
|
|
Œ t « } |j ||¬« }
|
|||
|
|
t d„ |D « « }|| |z
|
|||
|
|
z |
|
|||
|
|
d< t! |
|
|||
|
|
« S ) aà Evaluate a model's pipeline components.
|
|||
|
|
|
|||
|
|
examples (Iterable[Example]): `Example` objects.
|
|||
|
|
batch_size (Optional[int]): Batch size to use.
|
|||
|
|
scorer (Optional[Scorer]): Scorer to use. If not passed in, a new one
|
|||
|
|
will be created.
|
|||
|
|
component_cfg (dict): An optional dictionary with extra keyword
|
|||
|
|
arguments for specific components.
|
|||
|
|
scorer_cfg (dict): An optional dictionary with extra keyword arguments
|
|||
|
|
for the scorer.
|
|||
|
|
per_component (bool): Whether to return the scores keyed by component
|
|||
|
|
name. Defaults to False.
|
|||
|
|
|
|||
|
|
RETURNS (Scorer): The scorer containing the evaluation results.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#evaluate
|
|||
|
|
zLanguage.evaluatero c 3 ó4 K — | ] }|j –— Œ y wrµ rÔ rÖ s rk r~ z$Language.evaluate.<locals>.<genexpr>² s è ø€ Ð -¡H˜bˆR�\�\¡HùrØ )r� r¢ )r c 3 óF K — | ] }t |j « –— Œ y wrµ )rÍ rÕ rÖ s rk r~ z$Language.evaluate.<locals>.<genexpr>º s è ø€ Ð;±(¨B”c˜"Ÿ,™,×'±(ùs ‚ !Úspeedri )rÐ r= rÚ r� re rË r7 ÚtimerrÀ Ú referencer£ rý rà rÕ Úscorer… rá )rª rÏ r� r r¢ r r r« Ú
|
|||
|
|
start_timer× Údocsr¬ Úend_timeÚresultsÚn_wordss rk ÚevaluatezLanguage.evaluateƒ s# € ô6 ˜“>ˆÜ˜(Ð$7Ô8Ü! (Ó+ˆØ Ð Ø Ÿ ™ ˆJØ Ð Ø ˆMØ Ð ØˆJØ ˆ>ܘ*Ó%ˆFØ × Ñ ˜e TÔ *ÜÑ%˜fÑ%ˆFä“Wˆ
|
|||
|
|
ãˆBØ �M‰M˜"Ÿ,™,×+Ñ+Õ ,ð ð �y‰yÙ -¡HÓ -Ø!Ø'ð ó
|
|||
|
|
ˆô
|
|||
|
|
˜8 TÖ*‰GˆB�Ø ˆB�Lð +ä“7ˆØ—,‘,˜x°}�,ÓEˆÜÑ;±(Ó;Ó;ˆØ" h°Ñ&;Ñ<ˆ�ÑÜ$ WÓ-Ð-rj c óZ — d| j d d i}t j |« d S )zCCreate an optimizer, usually using the [training.optimizer] config.Ú optimizerr× )rQ rG r¥ )rª Ú subconfigs rk rÛ zLanguage.create_optimizer¾ s0 € à $§+¡+¨jÑ"9¸+Ñ"FÐGˆ Ü× Ñ Ó*¨;Ñ7Ð7rj Úparamsc # ó@ K — |sd–— y| j D ��cg c]. \ }}t |d« r t |d« r|j |« ‘Œ0 }}}|D ] } t |« Œ d–— |D ] } t |« Œ yc c}}w # t $ r Y Œ:w xY w# t $ r Y Œ2w xY ww)aÖ Replace weights of models in the pipeline with those provided in the
|
|||
|
|
params dictionary. Can be used as a contextmanager, in which case,
|
|||
|
|
models go back to their original weights after the block.
|
|||
|
|
|
|||
|
|
params (dict): A dictionary of parameters keyed by model ID.
|
|||
|
|
|
|||
|
|
EXAMPLE:
|
|||
|
|
>>> with nlp.use_params(optimizer.averages):
|
|||
|
|
>>> nlp.to_disk("/tmp/checkpoint")
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#use_params
|
|||
|
|
NÚ
|
|||
|
|
use_paramsr[ )r¹ r r, ÚnextÚ
|
|||
|
|
StopIteration)rª r* r‘ rý ÚcontextsrÆ s rk r, zLanguage.use_paramsà s² è ø€ ñ Ü ð #'§-¢-ôá"/‘J�D˜$ܘ4 Ô.´7¸4ÀÔ3Ið —‘ Õ'Ø"/ð
|
|||
|
|
ñ ó $�ðܘ•Mð $ó
|
|||
|
|
|
|||
|
|
Û#�ðܘ•Mñ $ùóøô %ò Ùðûô %ò ÙðüsX ‚B ™3A:Á B Á B Á B Á, BÁ7 B  B  B  B  B  BÂB ÂBÂB )Ú as_tuplesr� r¡ r¢ Ú n_processÚtextsr0 r1 c ó — y rµ ri ©rª r2 r0 r� r¡ r¢ r1 s rk rý z
|
|||
|
|
Language.pipeç ó € ð rj c ó — y rµ ri r4 s rk rý z
|
|||
|
|
Language.pipeô r5 rj r! c # ól ‡ K — |rst t t t t t
|
|||
|
|
f t f |« }ˆ fd„|D « }‰ j |||||¬« }|D ] } | j }
|
|||
|
|
d| _ | |
|
|||
|
|
f–— Œ yt t t t t
|
|||
|
|
f |« }|dk( rt j « }|€i }|€ ‰ j }g } ‰ j D ]g \ } }
|
|||
|
|
| |v rŒ |j | i « }|j d|« t j t" |
|
|||
|
|
| |‰ j$ ¬« }| j' |« Œi |dk7 rI‰ j) |« r#t+ j, t. j0 « ‰ j3 || ||« }nˆ fd„|D « }| D ]
|
|||
|
|
} ||« }Œ |D ] } | –— Œ yw) aX Process texts as a stream, and yield `Doc` objects in order.
|
|||
|
|
|
|||
|
|
texts (Iterable[Union[str, Doc]]): A sequence of texts or docs to
|
|||
|
|
process.
|
|||
|
|
as_tuples (bool): If set to True, inputs should be a sequence of
|
|||
|
|
(text, context) tuples. Output will then be a sequence of
|
|||
|
|
(doc, context) tuples. Defaults to False.
|
|||
|
|
batch_size (Optional[int]): The number of texts to buffer.
|
|||
|
|
disable (List[str]): Names of the pipeline components to disable.
|
|||
|
|
component_cfg (Dict[str, Dict]): An optional dictionary with extra keyword
|
|||
|
|
arguments for specific components.
|
|||
|
|
n_process (int): Number of processors to process texts. If -1, set `multiprocessing.cpu_count()`.
|
|||
|
|
YIELDS (Doc): Documents in the order of the original text.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#pipe
|
|||
|
|
c 3 óH •K — | ] \ }}‰j ||« –— Œ y wrµ )rÉ )r| r£ rÆ rª s €rk r~ z Language.pipe.<locals>.<genexpr> s' øè ø€ ð "ÙRWÁ ÀÀw�×-Ñ-¨d°G×<ÑRWùs ƒ ")r� r¡ r1 r¢ Néÿÿÿÿr� )r r‘ r« r© r! c 3 ó@ •K — | ] }‰j |« –— Œ y wrµ )r§ )r| r£ rª s €rk r~ z Language.pipe.<locals>.<genexpr>R s øè ø€ Ð=±u¨t�D×$Ñ$ T×*±uùs ƒ )r r r r rd r: rL rý rÈ ÚmpÚ cpu_countr� r¹ r¤ rË Ú functoolsÚpartialrD r© rÞ Ú_has_gpu_modelr] r^ r' ÚW114Ú_multiprocessing_pipe)rª r2 r0 r� r¡ r¢ r1 Údocs_with_contextsr" r¬ rÆ rú r‘ r r« Úfrý s ` rk rý z
|
|||
|
|
Language.pipe s¼ øè ø€ ñ8 Üœ¤%¬¬c´3¨h©¼Ð(DÑ"EÑFÈÓNˆEó"ÙRWó"Ð ð —9‘9Ø"Ø%Ø Ø#Ø +ð ó ˆDó �Ø Ÿ,™,�Ø #�” ؘG�nÓ$ð ð
|
|||
|
|
ä”Xœe¤C¬ H™oÑ.°Ó6ˆð ˜Š?ÜŸ ™ ›ˆIØ Ð Ø ˆMØ Ð Ø Ÿ ™ ˆJð
|
|||
|
|
ð ð Ÿ-œ-‰JˆD�$Ø�w‰ØØ"×&Ñ& t¨RÓ0ˆFà × Ñ ˜l¨JÔ 7Ü×!Ñ!ÜØØØ Ø&*×&@Ñ&@ô ˆAð
|
|||
|
|
�L‰L˜�Oð (𠘊>Ø×"Ñ" 7Ô+Ü—
|
|||
|
|
‘
|
|||
|
|
œhŸm™mÔ,à×-Ñ-¨e°U¸IÀzÓR‰Dó >±uÓ=ˆDÛ �Ù˜D“z‘ð ãˆCØ‹Iñ ùs ƒF1F4c ó — | j D ]p \ }}t |d« xr |j }||v s|sŒ't |d« sŒ4t |j d« sŒKt |j j
|
|||
|
|
t « sŒp y y)NrÞ r[ r
|
|||
|
|
TF)r¹ r rÞ r[ rŸ r
|
|||
|
|
r )rª r¡ r‘ r rÞ s rk r? zLanguage._has_gpu_modelX sm € Ø Ÿ-œ-‰JˆD�$Ü" 4¨ Ó8ÒN¸T×=NÑ=NˆLØ�w‰¡lØä�t˜WÕ%¬'°$·*±*¸eÕ*DÌÐTX×T^ÑT^×TbÑTbÔdkÕIlÙð
|
|||
|
|
(ð rj rú c
|
|||
|
|
# ód K — dt t t t f dt t t t t
|
|||
|
|
f t f fd„} ||« }t j |« \ }}t |« D �cg c] }t j « ‘Œ } }t t |« D �cg c] }t j d« ‘Œ c}Ž \ }
|
|||
|
|
} t j ||« } t! | | |¬« }
|
|||
|
|
|
|
|||
|
|
j# « |
|
|||
|
|
j# « t | | « D ��cg c]A \ }}t j$ t& | j( |||t+ j, « f¬« ‘ŒC }}}|D ] }|j/ « Œ | D ] }|j1 « Œ t3 j4 d„ t7 |
|
|||
|
|
« D « « } t9 t ||« d« D ]© \ }\ }\ }}}|�0t | j: « j= |« }||_ |–— nR|�PtA jB |« }| jE d d d tG tH jJ jM |¬ « « « ||z d
|
|||
|
|
k( sŒš|
|
|||
|
|
jO « Œ« | D ]' }|jQ tR « |j1 « Œ) |
|
|||
|
|
D ] }|j1 « Œ |D ] }|jU « Œ tW d „ |D « « s$tY jZ t\ j^ « y y c c}w c c}w c c}}w # | D ]' }|jQ tR « |j1 « Œ) |
|
|||
|
|
D ] }|j1 « Œ |D ] }|jU « Œ tW d „ |D « « s$tY jZ t\ j^ « w w xY ww) Nr2 rl c 3 ó¼ K — | D ]S }t |t « r-|j « t t |j
|
|||
|
|
« f–— Œ@|t t d « f–— ŒU y wrµ )rŸ r: Úto_bytesr rL rÈ )r2 rÁ s rk Ú
|
|||
|
|
prepare_inputz5Language._multiprocessing_pipe.<locals>.prepare_inputj sO è ø€ ó "�Ü ˜h¬Ô,Ø#×,Ñ,Ó.´´[À(×BSÑBSÓ0TÐUÓUà#¤T¬+°tÓ%<Ð=Ó=ñ "ùs ‚AA F)Ú
|
|||
|
|
chunk_size)Útargetr� c 3 ó< K — | ] }|j « –— Œ y wrµ )Úrecv)r| rL s rk r~ z1Language._multiprocessing_pipe.<locals>.<genexpr>¡ s è ø€ ð *
|
|||
|
|
Ù$;˜DˆD�I‰I�KÑ$;ùs ‚ r! r† r c 3 ó: K — | ] }|j d k( –— Œ yw)r N)Úexitcode)r| r s rk r~ z1Language._multiprocessing_pipe.<locals>.<genexpr>È s è ø€ Ð<±e¨d�t—}‘}¨Õ)±eùs ‚)0r r rd r: r rà rL Ú itertoolsÚteeÚranger; ÚQueuerà ÚPiper$ Ú minibatchÚ_SenderÚsendÚProcessÚ _apply_pipesrÉ r; Ú get_stateÚstartÚcloser Ú
|
|||
|
|
from_iterabler rÜ r{ rÄ rÈ rß Ú
|
|||
|
|
msgpack_loadsr© r r& ÚE871r¢ ÚstepÚputÚ_WORK_DONE_SENTINELrS Úallr] r^ r' ÚW127) rª r2 rú r1 r� rH Úserialized_texts_with_ctxÚ raw_textsrí Útexts_qÚbytedocs_recv_chÚbytedocs_send_chÚ batch_textsÚsenderÚrchÚschÚprocsr ÚtxÚ byte_tuplesr– Úbyte_docrÆ Ú
|
|||
|
|
byte_errorr¬ r‡ ÚqÚrs rk rA z Language._multiprocessing_pipec sE è ø€ ð
|
|||
|
|
>ÜœE¤#¤s (™OÑ,ð
|
|||
|
|
>ä
|
|||
|
|
”eœE¤#¤u *Ñ -¬{Ð :Ñ;Ñ
|
|||
|
|
<ó
|
|||
|
|
>ñ %2°%Ó$8Ð!ä$Ÿ=™=Ð)BÓCшˆyä7<¸YÔ7GÓ"HÑ7G°!¤2§8¡8¥:Ð7GˆÐ"Hä-0Ü&+¨IÔ&6Ó
|
|||
|
|
7Ñ&6 Œb�g‰g�e�nÐ&6Ñ
|
|||
|
|
7ð.
|
|||
|
|
Ñ*ÐÐ*ô —n‘n U¨JÓ7ˆ ô ˜ g¸)ÔDˆà� ‰ Œ
|
|||
|
|
Ø� ‰ Œ
|
|||
|
|
ô Ð)9Ô :ô
|
|||
|
|
ñ ;‘��Sô
|
|||
|
|
�J‰JÜ#à×1Ñ1ØØØÜ ×(Ñ(Ó*ð ö
|
|||
|
|
ð ;ð ñ
|
|||
|
|
ó ˆDØ �J‰J�Lð ó #ˆBØ �H‰H�Jð #ô
|
|||
|
|
×)Ñ)ñ *
|
|||
|
|
Ü$)Ð*:Ô$;ó*
|
|||
|
|
ó
|
|||
|
|
ˆ ð% -Ü;DÜ�I˜{Ó+¨Qö<Ñ7�Ñ7�AÑ6˜ '¨:ð Ð'Ü ˜dŸj™j›/×4Ñ4°XÓ>�CØ#*�C”LØ “IØ Ð+Ü !× /Ñ /°
|
|||
|
|
Ó ;�EØ×.Ñ.Ø ˜d D¬*´V·[±[×5GÑ5GÈeÐ5GÓ5TÓ*Uôð �z‘> QÓ&à—K‘K•Mñ <ó& �Ø—‘Ô)Ô*Ø—‘• ð ó &�Ø—‘• ð &ó �Ø— ‘ • ð ô Ñ<±eÓ<Ô<Ü—
|
|||
|
|
‘
|
|||
|
|
œhŸm™mÕ,ð =ùò] #Iùò 8ùó
|
|||
|
|
øó^ �Ø—‘Ô)Ô*Ø—‘• ð ó &�Ø—‘• ð &ó �Ø— ‘ • ð ô Ñ<±eÓ<Ô<Ü—
|
|||
|
|
‘
|
|||
|
|
œhŸm™mÕ,ð =üsN ‚A3N0Á5L ÂN0Â$ Là AN0ÄALÅ AN0Æ7B/L É'L É:B!N0ÌBN-Î-N0c ó. — | j D ] \ }}t |d« sŒ||_ Œ t | j « D ]U \ }\ }}t |t
|
|||
|
|
j « sŒ$i |_ | j |dz d D ] \ }}|j |« Œ ŒW y)zmRegister 'listeners' within pipeline components, to allow them to
|
|||
|
|
effectively share weights.
|
|||
|
|
r‘ r! N) r¹ r r‘ rÜ rŸ r# ÚListenedToComponentÚ listener_mapÚfind_listeners)rª r‘ r r– Úname1Úproc1Úname2Úproc2s rk ru zLanguage._link_componentsË sˆ € ð Ÿ-œ-‰JˆD�$Ü�t˜VÕ$Ø �• ð (ô "+¨4¯=©=Ö!9Ñ ˆA‰~��uܘ%¤×!7Ñ!7Õ8Ø%'�Ô"Ø$(§M¡M°!°a±%°'Ó$:‘L�E˜5Ø×(Ñ(¨Õ/ñ %;ñ ":rj )r{ r¡ r· rÍ r‹ Ú auto_fillrJ r| c ó8
|
|||
|
|
‡— |r*t | j t ¬« j |« }d|vr)t t
|
|||
|
|
j j |¬« « ‚d|d vr
|
|||
|
|
ddi|d d< |d j d« } | �]| | j k7 rNt t
|
|||
|
|
j j |d d | j t j | « ¬ « « ‚| j |d d< t j |« }|j d
|
|||
|
|
i « }
|
|||
|
|
|j d d« } i |d
|
|||
|
|
< |r t j ||t" ¬ « } n|} |
|
|||
|
|
| d
|
|||
|
|
< |
|
|||
|
|
|d
|
|||
|
|
< | �
|
|||
|
|
| | d < | |d < t j$ | d |t& ¬ « }
|
|||
|
|
|
|
|||
|
|
d
|
|||
|
|
}|
|
|||
|
|
d }|
|
|||
|
|
d }|
|
|||
|
|
d }|
|
|||
|
|
d }| }|�Z || « }t) |t* « rt- || « r|| ur2t t
|
|||
|
|
j. j t+ |« ¬« « ‚t1 « |||||¬« }|�G ||« }t) || « s3t t
|
|||
|
|
j2 j dt+ |« ¬« « ‚| j4 s| j7 « n| }|j d
|
|||
|
|
i « }i }i }d}|d d D �]^ }||vrIdj9 |j; « « }t t
|
|||
|
|
j< j ||¬« « ‚t j || « } t | d
|
|||
|
|
| « } |‰vsŒd| vr0d| vr,t
|
|||
|
|
j> j || ¬« } t | « ‚d| v r(| j d« } |jA | || || ¬« Œßd| v sJ ‚|€ |jB jE d d g¬ « }| d } | |vr&t jF | |jB d g¬ « || < | j d |« }!d!}"d"| v rU|| jI « || jJ D ]0 \ }#}$|!tM |$d#g « v sŒ|| jO |#|!| d" « d$}"Œ2 tQ jR « 5 tQ jT d%d&¬'« |jA |!|| |¬(« ddd« | |vr6tW || jB jX jE d g¬ « « || < d)|jZ vri |jZ d)< || |jZ d) |< |"s�Œ\|| = �Œa |�|jB j] |« t) |t^ « r|g}t) |t^ « r|g}t) ‰t^ « r‰gŠta |« ta tb « k7 rn|d j d*g « }%te |%« rNtg |%« ji |« s4tQ jj tl jn j ||%¬+« « | jq ts h |£|d j d,g « £« ||d d « }&tg ˆfd-„|&D « « |_: |d d. |_; |r| n||_< |�G ||« }t) || « s3t t
|
|||
|
|
j2 j d/t+ |« ¬« « ‚|S # 1 sw Y �ŒxY w)0a5 Create the nlp object from a loaded config. Will set up the tokenizer
|
|||
|
|
and language data, add pipeline components etc. If no config is provided,
|
|||
|
|
the default config of the given language is used.
|
|||
|
|
|
|||
|
|
config (Dict[str, Any] / Config): The loaded config.
|
|||
|
|
vocab (Vocab): A Vocab object. If True, a vocab is created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable.
|
|||
|
|
Disabled pipes will be loaded but they won't be run unless you
|
|||
|
|
explicitly enable them by calling nlp.enable_pipe.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
|
|||
|
|
pipes will be disabled (and can be enabled using `nlp.enable_pipe`).
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude.
|
|||
|
|
Excluded components won't be loaded.
|
|||
|
|
meta (Dict[str, Any]): Meta overrides for nlp.meta.
|
|||
|
|
auto_fill (bool): Automatically fill in missing values in config based
|
|||
|
|
on defaults and function argument annotations.
|
|||
|
|
validate (bool): Validate the component config and arguments against
|
|||
|
|
the types expected by the factory.
|
|||
|
|
RETURNS (Language): The initialized Language class.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#from_config
|
|||
|
|
rO ro rÚ r� z@vectorszspacy.Vectors.v1r N)Ú
|
|||
|
|
bad_lang_coderP r rÇ r )rJ rþ r“ Úbefore_creationÚafter_creationÚafter_pipeline_creation)rÓ )r{ r} rŒ r‹ Úcreation)r‘ rÓ r¹ rM rD rØ rY )r‘ rQ )r‘ rQ rJ rI rƒ rc ©rÍ )r{ rÍ r FÚreplace_listenersÚlistening_componentsTÚignorez\[W113\])Úmessage)rY r‘ rš Úenabled)r· rˆ rÈ c 3 ó, •K — | ] }|‰vsŒ|–— Œ
|
|||
|
|
y wrµ ri )r| rù rÍ s €rk r~ z'Language.from_config.<locals>.<genexpr>½ s øè ø€ ÐJ¡~ !¸À'Ò9IœA¡~ùs ƒ �r� Úpipeline_creation)=r r˜ rA r— r r& ÚE985r¢ r¤ r ÚE958r$ r? rb rU rG rT r2 r¥ r4 rŸ r£ Ú
|
|||
|
|
issubclassÚE943rH ÚE942Úis_interpolatedra rS rÄ ÚE956ÚE984ry r{ rG Ú
|
|||
|
|
load_modelru r¹ Úgetattrr„ r] Úcatch_warningsÚfilterwarningsÚhashr� r‹ rÄ rd Úidr@ rÍ rg Úissubsetr^ r' ÚW123Ú_resolve_component_statusrÐ r¨ r� rQ )'r² rQ r{ r¡ r· rÍ r‹ r| rJ Ú config_langÚ
|
|||
|
|
orig_pipelineÚorig_pretrainingrW Ú resolved_nlpr} rŒ r r€ r� Úlang_clsro Ú interpolatedr¹ Ú source_nlpsÚsource_nlp_vectors_hashesÚvocab_brâ rE Úpipe_cfgrI r* rØ r[ rX Úlisteners_replacedr‘ r rˆ Údisabled_pipess' ` rk Ú from_configzLanguage.from_configá så ø€ ñF ÜØ×"Ñ"Ô2Fôç‰e�F‹mð
|
|||
|
|
ð ˜Ñ Ü œVŸ[™[× /Ñ /°vÐ /Ó >Ó?Ð ?ð ˜F 5™MÑ )Ø(2Ð4FÐ'GˆF�5‰M˜)Ñ $Ø ˜U‘m×'Ñ'¨Ó/ˆ Ø Ð " {°c·h±hÒ'>Ü Ü— ‘ ×"Ñ"Ø"(¨¡-°Ñ"7Ø !Ÿh™hÜ ×-Ñ-¨cÓ2ð #ó óð
|
|||
|
|
ð !$§¡ˆˆu‰
|
|||
|
|
�fÑ ô
|
|||
|
|
×!Ñ! &Ó)ˆØ Ÿ
|
|||
|
|
™
|
|||
|
|
<°Ó4ˆ
|
|||
|
|
Ø!Ÿ:™: m°TÓ:ÐØ !ˆˆ|Ñ Ù Ü —]‘] 6°HÄ\ÔR‰FàˆFØ ,ˆˆ|Ñ Ø ,ˆˆ|Ñ Ø Ð 'Ø$4ˆF�=Ñ !Ø$4ˆF�=Ñ !Ü ×'Ñ'Ø �5‰M H´_ô
|
|||
|
|
ˆ ð (¨ Ñ4ÐØ% iÑ0ˆØ&Ð'8Ñ9ˆØ%Ð&6Ñ7ˆØ".Ð/HÑ"IÐ ØˆØ Ð &Ù& sÓ+ˆHä ˜x¬Ô.Ü! (¨CÔ0Ø 3Ñ&ä ¤§¡×!3Ñ!3¼$¸x».Ð!3Ó!IÓJÐJô Ô ñ ØØ -Ø)Øô
|
|||
|
|
ˆð Ð %Ù Ó%ˆCÜ ˜c 3Ô'Ü ¤§¡×!3Ñ!3¸Ì4ÐPSË9Ð!3Ó!UÓVÐVð 4:×3IÒ3I�v×)Ñ)Ô+Èvˆ Ø ×#Ñ# L°"Ó5ˆð ˆ Ø$&Ð!ØˆØ ™ zÕ2ˆIØ Ñ(Ø—y‘y § ¡ £ Ó1�Ü ¤§¡×!3Ñ!3¸ÈÐ!3Ó!NÓOÐOÜ×'Ñ'¨°Ñ(;Ó<ˆHÜ |Ñ 4°YÑ ?Ó@ˆJØ Ò'Ø HÑ,°ÀÑ1IÜ Ÿ+™+×,Ñ,°)ÀHÐ,ÓM�CÜ$ S›/Ð)Ø Ñ(Ø &Ÿl™l¨9Ó 5�Gð —L‘LØ Ø &Ø 'Ø!)Ø#-ð !õ ð $ xÑ/Ð/Ð/ð �Ø"%§)¡)×"4Ñ"4¸iÈÐ=SÐ"4Ó"T˜Ø $ XÑ .�EØ KÑ/ä-1¯_©_Ø !¨¯©¸Y¸Kô.˜ EÑ*ð #+§,¡,¨{¸IÓ"F�KØ).Ð&Ø*¨hÑ6ð $ EÑ*×;Ñ;Ô=Ø*5°eÑ*<×*EÔ*E™J˜D $Ø *¬g°dÐ<RÐTVÓ.WÒ WØ +¨EÑ 2× DÑ DØ$(¨+°xÐ@SÑ7Tô!"ð 6:Ñ 2ð +Fô "×0Ñ0Õ2Ü ×/Ñ/°À,ÕOØŸ ™ Ø '° ¸EÑ0BÈð %ô ÷ 3ð Ð$=Ñ=Ü;?Ø '¨Ñ .× 4Ñ 4× <Ñ <× EÑ EØ)2¨ ð Fó ó<Ð1°%Ñ8ð
|
|||
|
|
1¸¿¹Ñ@Ø>@˜Ÿ™Ð!:Ñ;ð 2°%Ñ8ð —H‘HÐ 6Ñ7Ø!ñó *Ø '¨Ò .ðQ 3ðT Ð Ø �I‰I× Ñ Ô )ô �gœsÔ #Ø �iˆGÜ �fœcÔ "Ø �XˆFÜ �gœsÔ #Ø �iˆGô ˆf‹:œÔ 0Ó1Ò 1Ø ˜U‘m×'Ñ'¨ °2Ó6ˆGÜ�7Œ|¤C¨£L×$9Ñ$9¸&Ô$AÜ—
|
|||
|
|
‘
|
|||
|
|
Ü —M‘M×(Ñ(Ø %Ø 'ð )ó ôð ×6Ñ6Ü Ð?�7Ð?˜V E™]× .Ñ .¨z¸2Ó >Ð?Ó @Ø Ø �5‰M˜*Ñ %ó
|
|||
|
|
ˆô
|
|||
|
|
ÓJ¡~ÓJÓJˆŒ
|
|||
|
|
à ™ |Ñ4ˆŒÙ (‘V¨fˆŒ
|
|||
|
|
Ø "Ð .Ù)¨#Ó.ˆCÜ ˜c 3Ô'Ü Ü—K‘K×&Ñ&Ð,?ÄtÈCÃyÐ&ÓQóð ð ˆ
|
|||
|
|
÷{ 3Ñ2ús Ñ/ZÚZ Ú tok2vec_namerâ Ú listenersc ó® — || j vrGt j j |||dj | j « ¬« }t |« ‚|| j vrGt j j |||dj | j « ¬« }t |« ‚| j
|
|||
|
|
|« }| j |« }t |t j « s3t t j j |t |« ¬« « ‚|j }|j j |g « }| j
|
|||
|
|
|« } | j | }
|
|||
|
|
|�r·t" j$ j' d|« t) t+ |« « t) |« k7 r7t j, j |||t) |« ¬« }t |« ‚|D ]d } t# j. |
|
|||
|
|
| « |d } d|j4 v r!|j4 d }
|
|||
|
|
|
|
|||
|
|
|d |
|
|||
|
|
d d « } t# j6 |
|
|||
|
|
| | « Œf |D ]Ñ }|j9 « }|j4 j d
|
|||
|
|
« }|�ot) t; j< |« j> « }|d k( r ||« }n9|d
|
|||
|
|
k( r ||||« }n)t t j@ j |¬« « ‚t# jB | j ||« |jE ||« ŒÓ y y # t0 $ r. t j2 j ||| ¬« }t |« ‚w xY w)aÅ Find listener layers (connecting to a token-to-vector embedding
|
|||
|
|
component) of a given pipeline component model and replace
|
|||
|
|
them with a standalone copy of the token-to-vector layer. This can be
|
|||
|
|
useful when training a pipeline with components sourced from an existing
|
|||
|
|
pipeline: if multiple components (e.g. tagger, parser, NER) listen to
|
|||
|
|
the same tok2vec component, but some of them are frozen and not updated,
|
|||
|
|
their performance may degrade significantly as the tok2vec component is
|
|||
|
|
updated with new data. To prevent this, listeners can be replaced with
|
|||
|
|
a standalone tok2vec layer that is owned by the component and doesn't
|
|||
|
|
change if the component isn't updated.
|
|||
|
|
|
|||
|
|
tok2vec_name (str): Name of the token-to-vector component, typically
|
|||
|
|
"tok2vec" or "transformer".
|
|||
|
|
pipe_name (str): Name of pipeline component to replace listeners for.
|
|||
|
|
listeners (Iterable[str]): The paths to the listeners, relative to the
|
|||
|
|
component config, e.g. ["model.tok2vec"]. Typically, implementations
|
|||
|
|
will only connect to one tok2vec component, [model.tok2vec], but in
|
|||
|
|
theory, custom models can use multiple listeners. The value here can
|
|||
|
|
either be an empty list to not replace any listeners, or a complete
|
|||
|
|
(!) list of the paths to all listener layers used by the model.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/language#replace_listeners
|
|||
|
|
rM )Útok2vecr‘ ÚunknownrE )r‘ rý z%Replacing listeners of component '%s')r‘ r¬ ÚpathsÚ n_listeners)r‘ r¬ r· r[ Úreplace_listener_cfgr¬ Úreplace_listenerNr! é )Ú
|
|||
|
|
num_params)#rÑ r& ÚE889r¢ rS r rH rÜ rŸ r# ru ÚE888r£ r[ rv r¤ rž r$ r� r‚ rÍ rÐ ÚE887Ú
|
|||
|
|
dot_to_objectrF ÚE886ÚattrsÚset_dot_to_objectÚcopyÚinspectÚ signatureÚ
|
|||
|
|
parametersÚE1055Úreplace_model_nodeÚremove_listener)rª r© râ rª r* r¬ Ú tok2vec_cfgÚ
|
|||
|
|
tok2vec_modelÚpipe_listenersrý r¥ Ú
|
|||
|
|
listener_pathÚ
|
|||
|
|
new_configÚ replace_funcÚlistenerÚ new_modelÚreplace_listener_funcr³ s rk r„ zLanguage.replace_listenersÉ s € ð: ˜tŸ™Ñ .Ü—+‘+×$Ñ$Ø$Ø Ø$Ø—Y‘Y˜tŸ™Ó/ð %ó ˆCô ˜S“/Ð !Ø ˜DŸO™OÑ +Ü—+‘+×$Ñ$Ø$Ø Ø!Ø—Y‘Y˜tŸ™Ó/ð %ó ˆCô ˜S“/Ð !Ø—-‘- Ó-ˆØ×*Ñ*¨<Ó8ˆ ܘ'¤2×#9Ñ#9Ô:Ü œVŸ[™[× /Ñ /°\ÌÈWË
|
|||
|
|
Ð /Ó VÓWÐ WØ Ÿ
|
|||
|
|
™
|
|||
|
|
ˆ
|
|||
|
|
Ø ×-Ñ-×1Ñ1°)¸RÓ@ˆØ�}‰}˜YÓ'ˆØ×%Ñ% iÑ0ˆÚ Ü �K‰K× Ñ Ð EÀyÔ QÜ”4˜ “?Ó#¤s¨>Ó':Ò:ô —k‘k×(Ñ(Ø"Ø (Ø#Ü # NÓ 3ð )ó �ô ! “oÐ%ó "+�
|
|||
|
|
ð*Ü×&Ñ& x° Ô?ð )¨Ñ 1�
|
|||
|
|
Ø)¨]×-@Ñ-@Ñ@Ø#0×#6Ñ#6Ð7MÑ#N�LÙ!-Ø# GÑ,¨h°wÑ.?À Ñ.Jó"�Jô ×&Ñ& x° À
|
|||
|
|
ÕKð "+ó" +�Ø )× .Ñ .Ó 0� Ø(5×(;Ñ(;×(?Ñ(?Ð@RÓ(SÐ%Ø(Ð4ô "%Ü ×)Ñ)Ð*?Ó@×KÑKó"�Jð " Q’Ù$9¸)Ó$D™ Ø# qš Ù$9¸)ÀXÈwÓ$W™ ä (¬¯ © ×)<Ñ)<È
|
|||
|
|
Ð)<Ó)SÓ TÐTä×'Ñ'¨¯
|
|||
|
|
©
|
|||
|
|
°H¸iÔHØ×'Ñ'¨°)Õ<ñ# +ðA øô& ò *Ü Ÿ+™+×,Ñ,Ø &° À=ð -ó �Cô % S›/Ð)ð *ús Ç"L Ì 7MÚmemc # óþ K — |€
|
|||
|
|
t « }t « 5 }|j | j j |« « g}t | j d« r9|j |j | j j |« « « | j D ]A \ }}t |d« sŒ|j |j |j |« « « ŒC |–— ddd« y# 1 sw Y yxY ww)a¢ Begin a block where all resources allocated during the block will
|
|||
|
|
be freed at the end of it. If a resources was created within the
|
|||
|
|
memory zone block, accessing it outside the block is invalid.
|
|||
|
|
Behaviour of this invalid access is undefined. Memory zones should
|
|||
|
|
not be nested.
|