714 lines
84 KiB
Plaintext
714 lines
84 KiB
Plaintext
|
Ë
|
|||
|
|
=û gæ ã ó’ — d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl Z d dl
|
|||
|
|
Z
|
|||
|
|
d dl Z d dl Z d dl
|
|||
|
|
Z
|
|||
|
|
d dlZd dlZd dlZd dlmZ d dlmZ d dlmZ d dlmZ d dlmZmZm Z m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z* d dl+Z+d dl,Z,d dl-Z-d dl.Z.d dl/Z/d dl+m0Z0m1Z1 d dl2m3Z3 d d l4m5Z5m6Z6 d d
|
|||
|
|
l7m8Z8m9Z9 d d l:m;Z;m<Z<m=Z=m>Z>m?Z?m@Z@mAZA d dlBZCd d l:mEZEmFZFmGZG d
|
|||
|
|
dlHmIZI d
|
|||
|
|
dlJmKZKmCZCmLZLmMZM d
|
|||
|
|
dlNmOZOmPZPmQZQ d
|
|||
|
|
dlRmSZS erd
|
|||
|
|
dlTmUZUmVZV d
|
|||
|
|
dlWmXZXmYZY d
|
|||
|
|
dlZm[Z[ e-j¸ e-jº « j¼ Z_dZ`g d¢Zag d¢Zb ejÆ d« Zd ejÊ « ZfefjÏ ejÐ d« « edjÓ ef« G d„ d« Zj G d „ d e/jÖ « Zk G d „ d el« Zm G d „ d!en« Zod"epd#eqfd$„Zrd"epd#e$ep fd%„Zsd"epd#e(d& fd'„Ztd(epd)e(d& d#dfd*„Zud+ed#efd,„Zvd+e)epef d#e)elenf fd-„Zwd.ed#efd/„Zx eo« Zyd0eyeyey em« d1œd(e)epef d2e)d3eqf d4e)epe ep f d5e)epe ep f d6e)epe ep f d7e)e epef e<f d#d&fd8„Zzd0eyeyey em« d1œd(epd2e)d3eqf d4e)epe ep f d5e)epe ep f d6e)epe ep f d7e)e epef e<f d#d&fd9„Z{dd0eyeyey em« d:œd;ed<e$e epef d2e)d3eqf d4e)epe ep f d5e)epe ep f d6e)epe ep f d7e)e epef e<f d#d&fd=„Z| em« d0eyeyeyd>d0d?œd7e)e epef e<f d<e epef d2e)d3eqf d4e)epe ep f d5e)epe ep f d6e)epe ep f d@eqdAeqd#d&fdB„Z}d7e)e epef e<f d#e epe epef f fdC„Z~d7e<dDe!e$ep d#e'edEf fdF„Zd0eyeyey em« d1œdGe)eepf d2e)d3eqf d4e)epe ep f d5e)epe ep f d6e)epe ep f d7e)e epef e<f d#d&fdH„Z€ em« d>fd+e)epef dIe epef dJeqd#e<fdK„Z� em« d>fdLepdIe epef dJeqfdM„Z‚d#e!ep fdN„Zƒd(epd#e$ep fdO„Z„ dÓdPepdQepdReqd#e$eq fdS„Z… dÓdQepdReqd#e$eq fdT„Z†dUepd#e'epepf fdV„Z‡dPepd#epfdW„ZˆdQepd#e$ep fdX„Z‰dPepd#eqfdY„ZŠdPepd#epfdZ„Z‹dPepd#e$ep fd[„ZŒd\epd]epd#eqfd^„Z�d+e)epef d#e epef fd_„ZŽd+e)epef d#e epef fd`„Z�d(epd#eqfda„Z�d(epd#efdb„Z‘dce>dde>dee>d#dfdf„Z’dgepd#e!ep fdh„Z“dd>diœdge)epe!ep f dje$e dkeqd#e
|
|||
|
|
�j( fdl„Z•ed+e)epef d#e e fdm„« Z–ed#e eddf fdn„« Z—d#eqfdo„Z˜d#eqfdp„Z™dqed#epfdr„Zšdse dte d#eqfdu„Z› dÔdveqdweqd#e$eK fdx„Zœdy„ Z�d+e)epef d#e%fdz„Zžd{e e)epe%f d#e%fd|„ZŸd{e e)epe%f d#e%fd}„Z d{e e)epe%f d#e%fd~„Z¡de epgef d#e epgef fd€„Z¢de epgef d�e epef d‚epd#efdƒ„Z£d„e epe!el f d#e epe!el f fd…„Z¤d†e epe!el f d‡epdˆepd#e epe!el f fd‰„Z¥ dÕdŠe¦d‹e¦dŒe¦d�e$e¦ d#e'e¦e¦f f
|
|||
|
|
dŽ„Z§d�e d� d#e!d� fd‘„Z¨d�e d� d#e!d� fd’„Z©ek�jU d“« d”„ « Z«d•e epe g e¬f f d6e ep d#e¬fd–„Zd—e¬d˜e epe e¬gef f d6e ep d#dfd™„Z®d•e epe g ef f d6e ep d#e epef fdš„Z¯d›e epef d˜e epe egef f d6e ep d#e epef fdœ„Z°d+e)epef d�e epe egdf f d6e ep d#efdž„Z±d+e)epef dŸe epe egdf f d6e ep d#efd „Z²d(epd¡e)epef d#efd¢„Z³d£epd#epfd¤„Z´dLepd#epfd¥„Zµd¦e ep dLepd#e'e!ep e!eq f fd§„Z¶d7e)e epef e<f d#e<fd¨„Z·d©e epef d#e epelf fdª„Z¸d>d«œdqe epelf d¬eqd#e epef fd„Z¹d7e<d®epfd¯„Zºd7e<d®epd°ed#dfd±„Z»g fd>d«œd²e epef d³e!ep d¬eqd#e e'e!ep ef fd´„Z¼dµe d#e!ep fd¶„Z½ em« fd·e!e epe$e¾ f dIe epe$e¾ f d#e epe$e¾ f fd¸„Z¿ G d¹„ dº« ZÀd#e@fd»„ZÁd¼„ ZÂdµe d#eqfd½„ZÃd¾epd#eqfd¿„ZÄdÀe dÁ dÂdÃd(epdÄe epdÃe!dÁ eÅge#f dÅe"epef d#e dÁ f dÆ„ZÆdÇ„ ZÇdÈ„ ZÈdÉ„ ZÉdÊ„ ZÊd#e¦fdË„ZËd#e epe!ep f fdÌ„ZÌdÍ„ ZÍdÖdÎe¦dÏepd#eqfdЄZÎd×d‹e¦dÏepdÑeqd#e¦fdÒ„ZÏy# eD$ r dZCY �Œyw xY w)Øé N)Ú defaultdict)Úcontextmanager)ÚPath)Ú
|
|||
|
|
ModuleType)Ú
|
|||
|
|
TYPE_CHECKINGÚAnyÚCallableÚDictÚ GeneratorÚIterableÚIteratorÚListÚMappingÚNoReturnÚOptionalÚPatternÚSetÚTupleÚTypeÚUnionÚcast)ÚRegistryÚ
|
|||
|
|
RegistryError)Ú Requirement)ÚInvalidSpecifierÚ SpecifierSet)ÚInvalidVersionÚVersion)ÚAdamÚConfigÚConfigValidationErrorÚModelÚNumpyOpsÚ OptimizerÚget_current_ops)Ú compoundingÚdecayingÚfix_random_seedé )Úabout)Ú
|
|||
|
|
CudaStreamÚcupyÚimportlib_metadataÚ
|
|||
|
|
is_windows)ÚOLD_MODEL_SHORTCUTSÚErrorsÚWarnings©ÚORTH)ÚLanguageÚ PipeCallable)ÚDocÚSpan)ÚVocabiìÿÿÿ)ÚcsÚdaÚdeÚelÚenÚgrcÚidÚlbÚmkÚptÚruÚsrÚtaÚth) ÚpathsÚ variablesÚsystemÚnlpÚ
|
|||
|
|
componentsÚcorporaÚtrainingÚ pretrainingÚ
|
|||
|
|
initializeÚspacyz)[%(asctime)s] [%(levelname)s] %(message)sc ó — e Zd ZdZy)ÚENV_VARSÚSPACY_CONFIG_OVERRIDESN)Ú__name__Ú
|
|||
|
|
__module__Ú __qualname__ÚCONFIG_OVERRIDES© ó úKC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/util.pyrR rR f s „ Ø/ÑrY rR c óŒ — e Zd Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej dd d¬« Z
|
|||
|
|
ej dd
|
|||
|
|
d¬« Z ej dd d¬« Z ej dd d¬« Z
|
|||
|
|
ej dd
|
|||
|
|
d¬« Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej ddd¬« Z ej dd« Z ej ddd¬« Z ej ddd¬« Zedee fd„« Zedededefd„« Z ededede ee e ee f f fd„« Z!ededede"fd „« Z#y ) ÚregistryrP Ú languagesT)Ú entry_pointsÚ
|
|||
|
|
architecturesÚ
|
|||
|
|
tokenizersÚ lemmatizersÚlookupsÚdisplacy_colorsÚmiscÚ callbacksÚbatchersÚreadersÚ
|
|||
|
|
augmentersÚloggersÚscorersÚvectorsÚ factoriesÚinternal_factoriesÚmodelsÚcliÚreturnc ó¾ — g }t j | « D ]9 \ }}|j d« rŒt |t « sŒ)|j |« Œ; t
|
|||
|
|
|« S )z List all available registries.Ú_)ÚinspectÚ
|
|||
|
|
getmembersÚ
|
|||
|
|
startswithÚ
|
|||
|
|
isinstancer ÚappendÚsorted)ÚclsÚnamesÚnameÚvalues rZ Úget_registry_nameszregistry.get_registry_names‡ sO € ð ˆÜ"×-Ñ-¨cÖ2‰KˆD�%Ø—?‘? 3Õ'¬J°u¼hÕ,GØ— ‘ ˜TÕ"ð 3ô �e‹}Ð rY Ú
|
|||
|
|
registry_nameÚ func_namec óz — t | |« sMdj | j « « xs d}t t j
|
|||
|
|
j
|
|||
|
|
||¬« « ‚t | |« } |j |« }|S # t $ r· |j d« r?|j dd« } |j |« cY S # t j $ r Y nw xY wdj t |j « j « « « xs d}t t j j
|
|||
|
|
|||¬« « d‚w xY w)z,Get a registered function from the registry.ú, Únone©r{ Ú availableúspacy.ú
|
|||
|
|
spacy-legacy.©r{ Úreg_namer„ N)ÚhasattrÚjoinr} r r0 ÚE892ÚformatÚgetattrÚgetru ÚreplaceÚ cataloguerx Úget_allÚkeysÚE893)ry r~ r rz ÚregÚfuncÚ legacy_namer„ s rZ rŽ z registry.get� s$ € ô
|
|||
|
|
�s˜MÔ*Ø—I‘I˜c× 4Ñ 4Ó 6Ó7ÒA¸6ˆEÜ ¤§ ¡ × 2Ñ 2¸
|
|||
|
|
ÐQVÐ 2Ó WÓXÐ XÜ�c˜=Ó)ˆð Ø—7‘7˜9Ó%ˆDð ˆ øô ò Ø×#Ñ# HÔ-Ø '× /Ñ /°¸/Ó J� ðØ Ÿ7™7 ;Ó/Ò/øÜ ×.Ñ.ò Ùðúà Ÿ ™ ¤&¨¯©« ×);Ñ);Ó)=Ó">Ó?ÒIÀ6ˆIÜ Ü— ‘ ×"Ñ"Ø"¨]Àið #ó óð ð
|
|||
|
|
ð úó1 Á'A: Á:,D:Â'B:Â7D:Â:CÃ
|
|||
|
|
D:ÃCÃA*D:c óz — t | |« sMdj | j « « xs d}t t j
|
|||
|
|
j
|
|||
|
|
||¬« « ‚t | |« } |j |« }|S # t $ r· |j d« r?|j dd« } |j |« cY S # t j $ r Y nw xY wdj t |j « j « « « xs d}t t j j
|
|||
|
|
|||¬« « d‚w xY w)au Find information about a registered function, including the
|
|||
|
|
module and path to the file it's defined in, the line number and the
|
|||
|
|
docstring, if available.
|
|||
|
|
|
|||
|
|
registry_name (str): Name of the catalogue registry.
|
|||
|
|
func_name (str): Name of the registered function.
|
|||
|
|
RETURNS (Dict[str, Optional[Union[str, int]]]): The function info.
|
|||
|
|
r� r‚ rƒ r… r† r‡ N)r‰ rŠ r} r r0 r‹ rŒ r� Úfindru r� r� rx r‘ r’ r“ )ry r~ r rz r” Ú func_infor– r„ s rZ r™ z
|
|||
|
|
registry.findª s% € ô �s˜MÔ*Ø—I‘I˜c× 4Ñ 4Ó 6Ó7ÒA¸6ˆEÜ ¤§ ¡ × 2Ñ 2¸
|
|||
|
|
ÐQVÐ 2Ó WÓXÐ XÜ�c˜=Ó)ˆð ØŸ™ Ó+ˆIð Ðøô ò Ø×#Ñ# HÔ-Ø '× /Ñ /°¸/Ó J� ðØ Ÿ8™8 KÓ0Ò0øÜ ×.Ñ.ò Ùðúà Ÿ ™ ¤&¨¯©« ×);Ñ);Ó)=Ó">Ó?ÒIÀ6ˆIÜ Ü— ‘ ×"Ñ"Ø"¨]Àið #ó óð ð
|
|||
|
|
ð úr— c ó– — t | |« syt | |« }|j d« r |j dd« }||v xs ||v S ||v S )z4Check whether a function is available in a registry.Fr… r† )r‰ r� ru r� )ry r~ r r” r– s rZ Úhasz registry.hasÍ s] € ô �s˜MÔ*ØÜ�c˜=Ó)ˆØ × Ñ Ô )Ø#×+Ñ+¨H°oÓFˆKØ Ð#Ò9 {°cÐ'9Ð 9ؘCÐ Ð rY N)$rT rU rV r� Úcreater] r_ r` ra rb rc rd re rf rg rh ri rj rk Ú_entry_point_factoriesrl rn ro Ú classmethodr Ústrr} r rŽ r
|
|||
|
|
r r Úintr™ Úboolrœ rX rY rZ r\ r\ j sH „ Ø � × Ñ ¨+ÀDÔI€IØ$�I×$Ñ$ W¨oÈDÔQ€MØ!�×!Ñ! '¨<ÀdÔK€JØ"�)×"Ñ" 7¨MÈÔM€KØ ˆi× Ñ ˜w¨ ÀÔE€GØ&�i×&Ñ& wÐ0AÐPTÔU€OØ ˆ9× Ñ ˜G V¸$Ô ?€Dà � × Ñ ¨+ÀDÔI€IØ ˆy× Ñ ¨À$ÔG€HØ ˆi× Ñ ˜w¨ ÀÔE€GØ!�×!Ñ! '¨<ÀdÔK€JØ ˆi× Ñ ˜w¨ ÀÔE€GØ ˆi× Ñ ˜w¨ ÀÔE€GØ ˆi× Ñ ˜w¨ ÀÔE€Gð .˜Y× -Ñ -¨g°{ÐQUÔ VÐØ � × Ñ Ð*>Ó?€Ið ˆY×
|
|||
|
|
Ñ
|
|||
|
|
˜g x¸dÔ
|
|||
|
|
C€FØ
|
|||
|
|
ˆ)×
|
|||
|
|
Ñ
|
|||
|
|
˜7 E¸Ô
|
|||
|
|
=€Càð 4¨¡9ò ó ð ð ð ð °ð ¸ò ó ðð2 ð Ø ð Ø,/ð à
|
|||
|
|
ˆc�8˜E # s (™OÑ,Ð,Ñ -ò ó ð ðD ð ð °ð ¸ò ó ñ rY r\ c óZ ‡ — e Zd ZdZej
|
|||
|
|
dœdeddfˆ fd„Zd„ Zd
|
|||
|
|
d„Z d „ Z
|
|||
|
|
ˆ xZ S ) ÚSimpleFrozenDictzâSimplified implementation of a frozen dict, mainly used as default
|
|||
|
|
function or method argument (for arguments that should default to empty
|
|||
|
|
dictionary). Will raise an error if user or spaCy attempts to add to dict.
|
|||
|
|
©Úerrorr¦ rp Nc ó2 •— t ‰| � |i |¤Ž || _ y)z£Initialize the frozen dict. Can be initialized with pre-defined
|
|||
|
|
values.
|
|||
|
|
|
|||
|
|
error (str): The error message when user tries to assign to dict.
|
|||
|
|
N)ÚsuperÚ__init__r¦ )Úselfr¦ ÚargsÚkwargsÚ __class__s €rZ r© zSimpleFrozenDict.__init__ß s ø€ ô ‰Ñ˜$Ð) &Ò)؈�
|
|||
|
|
rY c ó, — t | j « ‚©N©ÚNotImplementedErrorr¦ )rª Úkeyr| s rZ Ú __setitem__z SimpleFrozenDict.__setitem__è ó € Ü! $§*¡*Ó-Ð-rY c ó, — t | j « ‚r¯ r° )rª r² Údefaults rZ ÚpopzSimpleFrozenDict.popë r´ rY c ó, — t | j « ‚r¯ r° )rª Úothers rZ ÚupdatezSimpleFrozenDict.updateî r´ rY r¯ ) rT rU rV Ú__doc__r0 ÚE095r r© r³ r· rº Ú
|
|||
|
|
__classcell__©r s @rZ r¤ r¤ Ù s3 ø„ ñð
|
|||
|
|
,2¯;©;ò Sð ÀTõ ò.ó.ö.rY r¤ c óv ‡ — e Zd ZdZej
|
|||
|
|
dœdeddfˆ fd„Zd„ Zd„ Z d „ Z
|
|||
|
|
d
|
|||
|
|
„ Z d „ Z d „ Z
|
|||
|
|
d
|
|||
|
|
„ Zd„ Zˆ xZS )ÚSimpleFrozenLista‚ Wrapper class around a list that lets us raise custom errors if certain
|
|||
|
|
attributes/methods are accessed. Mostly used for properties like
|
|||
|
|
Language.pipeline that return an immutable list (and that we don't want to
|
|||
|
|
convert to a tuple to not break too much backwards compatibility). If a user
|
|||
|
|
accidentally calls nlp.pipeline.append(), we can raise a more helpful error.
|
|||
|
|
r¥ r¦ rp Nc ó, •— || _ t ‰| � |Ž y)zpInitialize the frozen list.
|
|||
|
|
|
|||
|
|
error (str): The error message when user tries to mutate the list.
|
|||
|
|
N)r¦ r¨ r© )rª r¦ r« r s €rZ r© zSimpleFrozenList.__init__ú s ø€ ð
|
|||
|
|
ˆŒ
|
|||
|
|
Ü
|
|||
|
|
‰Ñ˜$Ò rY c ó, — t | j « ‚r¯ r° ©rª r« r¬ s rZ rw zSimpleFrozenList.append r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚclearzSimpleFrozenList.clear r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚextendzSimpleFrozenList.extend r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚinsertzSimpleFrozenList.insert r´ rY c ó, — t | j « ‚r¯ r° rà s rZ r· zSimpleFrozenList.pop r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚremovezSimpleFrozenList.remove r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚreversezSimpleFrozenList.reverse r´ rY c ó, — t | j « ‚r¯ r° rà s rZ ÚsortzSimpleFrozenList.sort r´ rY )rT rU rV r» r0 ÚE927r r© rw rÅ rÇ rÉ r· rÌ rÎ rÐ r½ r¾ s @rZ rÀ rÀ ò sL ø„ ñð ,2¯;©;ò Sð ¸4õ ò.ò.ò.ò.ò.ò.ò.ö.rY rÀ Úlangrp c ó& — | t j v S )a Check whether a Language class is already loaded. Language classes are
|
|||
|
|
loaded lazily, to avoid expensive setup code associated with the language
|
|||
|
|
data.
|
|||
|
|
|
|||
|
|
lang (str): Two-letter language code, e.g. 'en'.
|
|||
|
|
RETURNS (bool): Whether a Language class has been loaded.
|
|||
|
|
)r\ r] )rÒ s rZ Úlang_class_is_loadedrÔ s € ð ”8×%Ñ%Ð %Ð%rY c óJ — ddl }| dk( ryg }t j |j j « D ]L }|j
|
|||
|
|
}|dk( r|j
|
|||
|
|
d« Œ&t j |« sŒ<|j
|
|||
|
|
|« ŒN t j | |d¬« }|dk( ry|S )a‚
|
|||
|
|
Given an IETF language code, find a supported spaCy language that is a
|
|||
|
|
close match for it (according to Unicode CLDR language-matching rules).
|
|||
|
|
This allows for language aliases, ISO 639-2 codes, more detailed language
|
|||
|
|
tags, and close matches.
|
|||
|
|
|
|||
|
|
Returns the language code if a matching language is available, or None
|
|||
|
|
if there is no matching language.
|
|||
|
|
|
|||
|
|
>>> find_matching_language('en')
|
|||
|
|
'en'
|
|||
|
|
>>> find_matching_language('pt-BR') # Brazilian Portuguese
|
|||
|
|
'pt'
|
|||
|
|
>>> find_matching_language('fra') # an ISO 639-2 code for French
|
|||
|
|
'fr'
|
|||
|
|
>>> find_matching_language('iw') # obsolete alias for Hebrew
|
|||
|
|
'he'
|
|||
|
|
>>> find_matching_language('no') # Norwegian
|
|||
|
|
'nb'
|
|||
|
|
>>> find_matching_language('mo') # old code for ro-MD
|
|||
|
|
'ro'
|
|||
|
|
>>> find_matching_language('zh-Hans') # Simplified Chinese
|
|||
|
|
'zh'
|
|||
|
|
>>> find_matching_language('zxx')
|
|||
|
|
None
|
|||
|
|
r NÚxxÚmulé )Ú max_distance)
|
|||
|
|
Ú
|
|||
|
|
spacy.langÚpkgutilÚ iter_modulesrÒ Ú__path__r{ rw Ú langcodesÚ tag_is_validÚclosest_supported_match)rÒ rP Úpossible_languagesÚmodinfoÚcodeÚmatchs rZ Úfind_matching_languagerå & s™ € ó6 à ˆt‚|Øð ÐÜ×'Ñ'¨¯
|
|||
|
|
©
|
|||
|
|
×(;Ñ(;Ö<ˆØ�|‰|ˆØ �4Š<à × %Ñ % eÕ ,Ü
|
|||
|
|
×
|
|||
|
|
#Ñ
|
|||
|
|
# DÕ
|
|||
|
|
)Ø × %Ñ % dÕ +ð
|
|||
|
|
=ô
|
|||
|
|
× -Ñ -¨dÐ4FÐUVÔ W€EØ �‚~ààˆ rY r4 c ó> — | t j v r t j j | « S t j d| › �d« }t | t ||j d « « t j j | « S # t
|
|||
|
|
$ r…} t
|
|||
|
|
| « }n%# t j j $ r d}Y nw xY w|r |} t j d| › �d« }n+t t j j | |¬« « |‚Y d}~ŒËd}~ww xY w)z�Import and load a Language class.
|
|||
|
|
|
|||
|
|
lang (str): IETF language code, such as 'en'.
|
|||
|
|
RETURNS (Language): Language class.
|
|||
|
|
z.lang.rP N)rÒ Úerrr )r\ r] rŽ Ú importlibÚ
|
|||
|
|
import_moduleÚ ImportErrorrå rÞ Ú
|
|||
|
|
tag_parserÚLanguageTagErrorr0 ÚE048rŒ Úset_lang_classr� Ú__all__)rÒ Úmodulerç rä s rZ Úget_lang_classrñ ] s € ð Œx×!Ñ!Ñ!Ü×!Ñ!×%Ñ% dÓ+Ð+ð SÜ ×,Ñ,¨v°d°V¨_¸gÓFˆFô �tœW V¨V¯^©^¸AÑ->Ó ?Ô@Ü × Ñ × !Ñ ! $Ó 'Ð'øô ò
|
|||
|
|
Sð
|
|||
|
|
Ü.¨tÓ4‘øÜ ×'Ñ'×8Ñ8ò
|
|||
|
|
à ’ð
|
|||
|
|
úñ Ø �Ü"×0Ñ0°6¸$¸° À'ÓJ‘ä!¤&§+¡+×"4Ñ"4¸$ÀCÐ"4Ó"HÓIÈsÐRô ûð
|
|||
|
|
Sús6 ³B  D  B$Â#DÂ$ CÃDÃCÃA DÄD r{ ry c óF — t j j | |¬« y)z™Set a custom Language class name that can be loaded via get_lang_class.
|
|||
|
|
|
|||
|
|
name (str): Name of Language class.
|
|||
|
|
cls (Language): Language class.
|
|||
|
|
)r• N)r\ r] Úregister)r{ ry s rZ rî rî | s € ô
|
|||
|
|
×Ñ× Ñ ¨3Ð Õ/rY Úpathc ó< — t | t « r t | « S | S )z�Ensure string is converted to a Path.
|
|||
|
|
|
|||
|
|
path (Any): Anything. If string, it's converted to Path.
|
|||
|
|
RETURNS: Path or original argument.
|
|||
|
|
)rv r r ©rô s rZ Ú ensure_pathr÷ … s € ô �$œÔ Ü�D‹zÐàˆ rY c ó: — t | « } | j « rt j | « S | j | j
|
|||
|
|
dz « } | j « rt j | « S t t j j | ¬« « ‚)zåLoad JSON language data using the given path as a base. If the provided
|
|||
|
|
path isn't present, will attempt to load a gzipped version before giving up.
|
|||
|
|
|
|||
|
|
path (str / Path): The data to load.
|
|||
|
|
RETURNS: The loaded data.
|
|||
|
|
z.gzrö ) r÷ ÚexistsÚsrslyÚ read_jsonÚ with_suffixÚsuffixÚread_gzip_jsonÚ
|
|||
|
|
ValueErrorr0 ÚE160rŒ rö s rZ Úload_language_datar ‘ sv € ô �tÓ €DØ ‡{�{„}Ü�‰˜tÓ$Ð$Ø × Ñ ˜DŸK™K¨%Ñ /Ó 0€DØ ‡{�{„}Ü×#Ñ# DÓ)Ð)Ü
|
|||
|
|
”V—[‘[×'Ñ'¨TÐ'Ó2Ó
|
|||
|
|
3Ð3rY rð c ó& — t | d« s2t t j j t | « ¬« « ‚t
|
|||
|
|
t t j t j | j j « « }|j S )zpGet the path of a Python module.
|
|||
|
|
|
|||
|
|
module (ModuleType): The Python module.
|
|||
|
|
RETURNS (Path): The path.
|
|||
|
|
rU )rð )r‰ rÿ r0 ÚE169rŒ Úreprr r ÚosÚPathLikeÚsysÚmodulesrU Ú__file__Úparent)rð Ú file_paths rZ Úget_module_pathr ¡ se € ô �6˜<Ô (ÜœŸ™×+Ñ+´4¸³<Ð+Ó@ÓAÐAÜ”Tœ"Ÿ+™+¤s§{¡{°6×3DÑ3DÑ'E×'NÑ'NÓOÓP€IØ × Ñ ÐrY T©ÚvocabÚdisableÚenableÚexcludeÚconfigr r8 r r r r c ó — |||||dœ}t | t « rv| j d« r t | j dd« « « S t | « r t
|
|||
|
|
| fi |¤ŽS t | « j « r-t t | « fi |¤ŽS t | d« r t | fi |¤ŽS | t v r1t t j j | t | ¬« « ‚t t j j | ¬« « ‚)a£ Load a model from a package or data path.
|
|||
|
|
|
|||
|
|
name (str): Package name or model path.
|
|||
|
|
vocab (Vocab / True): Optional vocab to pass in on initialization. If True,
|
|||
|
|
a new Vocab object will be created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All others will be disabled.
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude.
|
|||
|
|
config (Dict[str, Any] / Config): Config overrides as nested dict or dict
|
|||
|
|
keyed by section values in dot notation.
|
|||
|
|
RETURNS (Language): The loaded nlp object.
|
|||
|
|
r
|
|||
|
|
zblank:Ú rù )r{ Úfull©r{ )rv r ru rñ r� Ú
|
|||
|
|
is_packageÚload_model_from_packager rù Úload_model_from_pathr‰ r/ ÚIOErrorr0 ÚE941rŒ ÚE050)r{ r r r r r r¬ s rZ Ú
|
|||
|
|
load_modelr ± sî € ð, ØØØØñ €Fô �$œÔ Ø �?‰?˜8Ô $Ø=”> $§,¡,¨x¸Ó"<Ó=Ó?Ð ?Ü �dÔ Ü*¨4Ñ:°6Ñ:Ð :Ü �‹:× Ñ Ô Ü'¬¨T«
|
|||
|
|
Ñ=°fÑ=Ð =Ü ��xÔ Ü# DÑ3¨FÑ3Ð3Ø Ô"Ñ"Ü”f—k‘k×(Ñ(¨dÔ9LÈTÑ9RÐ(ÓSÓTÐTÜ
|
|||
|
|
”&—+‘+×$Ñ$¨$Ð$Ó/Ó
|
|||
|
|
0Ð0rY c óX — t j | « }|j |||||¬« S )a Load a model from an installed package.
|
|||
|
|
|
|||
|
|
name (str): The package name.
|
|||
|
|
vocab (Vocab / True): Optional vocab to pass in on initialization. If True,
|
|||
|
|
a new Vocab object will be created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
|
|||
|
|
pipes will be loaded but they won't be run unless you explicitly
|
|||
|
|
enable them by calling nlp.enable_pipe.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
|
|||
|
|
pipes will be disabled (and can be enabled using `nlp.enable_pipe`).
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
|
|||
|
|
components won't be loaded.
|
|||
|
|
config (Dict[str, Any] / Config): Config overrides as nested dict or dict
|
|||
|
|
keyed by section values in dot notation.
|
|||
|
|
RETURNS (Language): The loaded nlp object.
|
|||
|
|
r
|
|||
|
|
)rè ré Úload)r{ r r r r r ry s rZ r r Û s. € ô2 ×
|
|||
|
|
!Ñ
|
|||
|
|
! $Ó
|
|||
|
|
'€CØ �8‰8˜%¨¸ÈÐY_ˆ8Ó `Ð`rY )Úmetar r r r r Ú
|
|||
|
|
model_pathr c ó — | j « s)t t j j | ¬« « ‚|s t | « }| dz }t
|
|||
|
|
|d¬« }t ||¬« }t ||||||¬« } | j | ||¬« S )a Load a model from a data directory path. Creates Language class with
|
|||
|
|
pipeline from config.cfg and then calls from_disk() with path.
|
|||
|
|
|
|||
|
|
model_path (Path): Model path.
|
|||
|
|
meta (Dict[str, Any]): Optional model meta.
|
|||
|
|
vocab (Vocab / True): Optional vocab to pass in on initialization. If True,
|
|||
|
|
a new Vocab object will be created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
|
|||
|
|
pipes will be loaded but they won't be run unless you explicitly
|
|||
|
|
enable them by calling nlp.enable_pipe.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
|
|||
|
|
pipes will be disabled (and can be enabled using `nlp.enable_pipe`).
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
|
|||
|
|
components won't be loaded.
|
|||
|
|
config (Dict[str, Any] / Config): Config overrides as nested dict or dict
|
|||
|
|
keyed by section values in dot notation.
|
|||
|
|
RETURNS (Language): The loaded nlp object.
|
|||
|
|
rö z
|
|||
|
|
config.cfgT©Ú
|
|||
|
|
for_overrides)Ú overrides)r r r r r )r r% )
|
|||
|
|
rù r r0 ÚE052rŒ Úget_model_metaÚ dict_to_dotÚ load_configÚload_model_from_configÚ from_disk)
|
|||
|
|
r! r r r r r r Ú config_pathr% rJ s
|
|||
|
|
rZ r r ø s� € ð8 × Ñ Ô Ü”f—k‘k×(Ñ(¨jÐ(Ó9Ó:Ð:Ù Ü ˜jÓ)ˆØ ˜|Ñ+€KܘF°$Ô7€IÜ
|
|||
|
|
˜° Ô
|
|||
|
|
:€FÜ
|
|||
|
|
ØØØØØØ
|
|||
|
|
ô
|
|||
|
|
€Cð �=‰=˜¨WÀ ˆ=Ó JÐJrY F)r r r r r Ú auto_fillÚvalidater- r. c
|
|||
|
|
ó — d| vr)t t j j | ¬« « ‚| d }d|vs|d €)t t j j |¬« « ‚t |d « } | j
|
|||
|
|
| |||||||¬« }
|
|||
|
|
|
|
|||
|
|
S )a% Create an nlp object from a config. Expects the full config file including
|
|||
|
|
a section "nlp" containing the settings for the nlp object.
|
|||
|
|
|
|||
|
|
name (str): Package name or model path.
|
|||
|
|
meta (Dict[str, Any]): Optional model meta.
|
|||
|
|
vocab (Vocab / True): Optional vocab to pass in on initialization. If True,
|
|||
|
|
a new Vocab object will be created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
|
|||
|
|
pipes will be loaded but they won't be run unless you explicitly
|
|||
|
|
enable them by calling nlp.enable_pipe.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
|
|||
|
|
pipes will be disabled (and can be enabled using `nlp.enable_pipe`).
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
|
|||
|
|
components won't be loaded.
|
|||
|
|
auto_fill (bool): Whether to auto-fill config with missing defaults.
|
|||
|
|
validate (bool): Whether to show config validation errors.
|
|||
|
|
RETURNS (Language): The loaded nlp object.
|
|||
|
|
rJ ©r rÒ )r r r r r- r. r )rÿ r0 ÚE985rŒ ÚE993rñ Ú from_config) r r r r r r r- r. Ú
|
|||
|
|
nlp_configÚlang_clsrJ s rZ r* r* & s¦ € ð:
|
|||
|
|
�FÑÜœŸ™×+Ñ+°6Ð+Ó:Ó;Ð;ؘ‘ €JØ
|
|||
|
|
�ZÑ :¨fÑ#5Ð#=ÜœŸ™×+Ñ+°:Ð+Ó>Ó?Ð?ô ˜j¨Ñ 0Ó1€HØ
|
|||
|
|
×
|
|||
|
|
Ñ
|
|||
|
|
ØØØØØØØØ
|
|||
|
|
ð ó €Cð €JrY c ó† — | j di « j « D ��ci c] \ }}d|vrd|v r||“Œ c}}S c c}}w )zÇRETURNS (List[str]): All sourced components in the original config,
|
|||
|
|
e.g. {"source": "en_core_web_sm"}. If the config contains a key
|
|||
|
|
"factory", we assume it refers to a component factory.
|
|||
|
|
rK ÚfactoryÚsource)rŽ Úitems)r r{ Úcfgs rZ Úget_sourced_componentsr; X sS € ð Ÿ™ L°"Ó5×;Ñ;Ô=ô á=‰IˆD�#Ø ˜CÑ H°¡Oð
|
|||
|
|
ˆc‰ Ø=ò ð ùó s ¤=Ú dot_names.c óô — i }g }g }|D ]� }|€|j |« Œ|j d« d }||vrSt j | | « r t j d| | i« d }nt j | | « }|||< |j t ||« « ŒŸ |r
|
|||
|
|
t | |¬« ‚t |« S # t $ r+ d|› �}|j |j d« |dœ« Y Œíw xY w)a: Resolve one or more "dot notation" names, e.g. corpora.train.
|
|||
|
|
The paths could point anywhere into the config, so we don't know which
|
|||
|
|
top-level section we'll be looking within.
|
|||
|
|
|
|||
|
|
We resolve the whole top-level section, although we could resolve less --
|
|||
|
|
we could find the lowest part of the tree.
|
|||
|
|
ú.r r z not a valid section reference: )ÚlocÚmsg)r Úerrors) rw Úsplitr\ Ú
|
|||
|
|
is_promiseÚresolveÚ
|
|||
|
|
dot_to_objectÚKeyErrorr! Útuple) r r< ÚresolvedÚoutputrA r{ ÚsectionÚresultr@ s rZ Úresolve_dot_namesrL f s € ð €HØ€FØ
|
|||
|
|
€FÛˆØ ˆ<Ø �M‰M˜$Õ à—j‘j “o aÑ(ˆGà˜hÑ&Ü×&Ñ& v¨g¡Ô7ä %× -Ñ -¨x¸À¹ Ð.IÓ JÈ8Ñ T‘Fä %× -Ñ -¨f°W©oÓ >�FØ$*�˜Ñ!ð
|
|||
|
|
DØ—
|
|||
|
|
‘
|
|||
|
|
œm¨H°dÓ ;Õ<ð ñ$ Ü#¨6¸&ÔAÐAÜ �‹=Ðøô ò
|
|||
|
|
DØ7¸°vÐ>�Ø—
|
|||
|
|
‘
|
|||
|
|
d§j¡j°£o¸cÑ BÖCð
|
|||
|
|
Dús  CÃ1C7Ã6C7Ú init_filec ó — t | « j }t |« }|d › d|d › d|d › �}||z } |j « s)t t
|
|||
|
|
j j | ¬« « ‚t | ||||||¬« S )a‰ Helper function to use in the `load()` method of a model package's
|
|||
|
|
__init__.py.
|
|||
|
|
|
|||
|
|
vocab (Vocab / True): Optional vocab to pass in on initialization. If True,
|
|||
|
|
a new Vocab object will be created.
|
|||
|
|
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
|
|||
|
|
pipes will be loaded but they won't be run unless you explicitly
|
|||
|
|
enable them by calling nlp.enable_pipe.
|
|||
|
|
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
|
|||
|
|
pipes will be disabled (and can be enabled using `nlp.enable_pipe`).
|
|||
|
|
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
|
|||
|
|
components won't be loaded.
|
|||
|
|
config (Dict[str, Any] / Config): Config overrides as nested dict or dict
|
|||
|
|
keyed by section values in dot notation.
|
|||
|
|
RETURNS (Language): The loaded nlp object.
|
|||
|
|
rÒ rr r{ ú-Úversionrö )r r r r r r ) r r
|
|||
|
|
r' rù r r0 r& rŒ r )
|
|||
|
|
rM r r r r r r! r Údata_dirÚ data_paths
|
|||
|
|
rZ Úload_model_from_init_pyrS ‹ s• € ô2 �i“ ×'Ñ'€JÜ ˜*Ó %€DØ�v‘,� ˜q f¡ ¨a°°Y±Ð/@ÐA€HؘXÑ%€IØ × Ñ Ô Ü”f—k‘k×(Ñ(¨iÐ(Ó8Ó9Ð9Ü ØØØ
|
|||
|
|
ØØØØô ð rY r% Ú interpolatec óT — t | « }t t ¬« }t |« dk( r0|j t
|
|||
|
|
j j « ||¬« S |r|j « s*t t j j |d¬« « ‚|j |||¬« S )aƒ Load a config file. Takes care of path validation and section order.
|
|||
|
|
|
|||
|
|
path (Union[str, Path]): Path to the config file or "-" to read from stdin.
|
|||
|
|
overrides: (Dict[str, Any]): Config overrides as nested dict or
|
|||
|
|
dict keyed by section values in dot notation.
|
|||
|
|
interpolate (bool): Whether to interpolate and resolve variables.
|
|||
|
|
RETURNS (Config): The loaded config.
|
|||
|
|
©Ú
|
|||
|
|
section_orderrO ©r% rT z config file©rô r{ )r÷ r ÚCONFIG_SECTION_ORDERr Úfrom_strr ÚstdinÚreadÚis_filer r0 ÚE053rŒ r+ )rô r% rT r, r s rZ r) r) µ sœ € ô ˜dÓ#€KÜ
|
|||
|
|
Ô"6Ô
|
|||
|
|
7€FÜ
|
|||
|
|
ˆ;Ó˜3Ò Ø�‰Ü �I‰I�N‰NÓ ¨ ¸{ð ó
|
|||
|
|
ð
|
|||
|
|
ñ +×"5Ñ"5Ô"7Üœ&Ÿ+™+×,Ñ,°+ÀMÐ,ÓRÓSÐ SØ× Ñ Ø 9¸+ð ó
|
|||
|
|
ð
|
|||
|
|
rY Útextc óF — t t ¬« j | ||¬« S )zéLoad a full config from a string. Wrapper around Thinc's Config.from_str.
|
|||
|
|
|
|||
|
|
text (str): The string config to load.
|
|||
|
|
interpolate (bool): Whether to interpolate and resolve variables.
|
|||
|
|
RETURNS (Config): The loaded config.
|
|||
|
|
rV rX )r rZ r[ )r` r% rT s rZ Úload_config_from_strrb Ð s) € ô Ô 4Ô 5× >Ñ >Ø ˜ ¨{ð ?ó ð rY c ól — t t j j « j « « S )z~List all model packages currently installed in the environment.
|
|||
|
|
|
|||
|
|
RETURNS (List[str]): The string names of the models.
|
|||
|
|
)Úlistr\ rn r‘ r’ rX rY rZ Úget_installed_modelsre Þ s% € ô
|
|||
|
|
”—‘×'Ñ'Ó)×.Ñ.Ó0Ó 1Ð1rY c ó` — t j | « S # t j $ r Y yw xY w)zâGet the version of an installed package. Typically used to get model
|
|||
|
|
package versions.
|
|||
|
|
|
|||
|
|
name (str): The name of the installed Python package.
|
|||
|
|
RETURNS (str / None): The version or None if package not installed.
|
|||
|
|
N)r- rP ÚPackageNotFoundErrorr s rZ Úget_package_versionrh æ s0 € ðÜ!×)Ñ)¨$Ó/Ð/øÜ × 2Ñ 2ò Ùðús ‚ —-¬-rP Ú
|
|||
|
|
constraintÚ prereleasesc ó — |d j « rd|› �} t |« }t | « } ||_ | |v S # t t f$ r Y yw xY w)a
|
|||
|
|
Check if a version (e.g. "2.0.0") is compatible given a version
|
|||
|
|
constraint (e.g. ">=1.9.0,<2.2.1"). If the constraint is a specific version,
|
|||
|
|
it's interpreted as =={version}.
|
|||
|
|
|
|||
|
|
version (str): The version to check.
|
|||
|
|
constraint (str): The constraint string.
|
|||
|
|
prereleases (bool): Whether to allow prereleases. If set to False,
|
|||
|
|
prerelease versions will be considered incompatible.
|
|||
|
|
RETURNS (bool / None): Whether the version is compatible, or None if the
|
|||
|
|
version or constraint are invalid.
|
|||
|
|
r ú==N)Úisdigitr r r r rj )rP ri rj Úspecs rZ Úis_compatible_versionro ó sf € ð �!�}× Ñ Ô Ø˜*˜ Ð&ˆ
|
|||
|
|
ðܘJÓ'ˆÜ˜'Ó"ˆð #€DÔØ �dˆ?Ðøô
|
|||
|
|
œnÐ -ò Ùðús š; »A
|
|||
|
|
Á A
|
|||
|
|
c óP — | d j « ry t | « }||_ |D �cg c] }|‘Œ }}t |« dk( r|d j
|
|||
|
|
dv ryt
|
|||
|
|
d„ |D « « ryt
|
|||
|
|
d„ |D « « }t
|
|||
|
|
d„ |D « « }|r|ryy# t $ r Y y w xY wc c}w ) Nr Fr) ©ú>ú>=Tc 3 ó8 K — | ] }|j d v –— Œ yw)rl N©Úoperator©Ú.0Úsps rZ ú <genexpr>z+is_unconstrained_version.<locals>.<genexpr> s è ø€ Ð
|
|||
|
|
1©5 Rˆ2�;‰;˜4Ô ©5ùó ‚c 3 ó8 K — | ] }|j d v –— Œ yw))ú<z<=Nru rw s rZ rz z+is_unconstrained_version.<locals>.<genexpr> ó è ø€ Ð?¹°2�B—K‘K ;Ô.¹ùr{ c 3 ó8 K — | ] }|j d v –— Œ yw)rq Nru rw s rZ rz z+is_unconstrained_version.<locals>.<genexpr> r~ r{ )rm r r rj Úlenrv Úany)ri rj rn ry ÚspecsÚ has_upperÚ has_lowers rZ Úis_unconstrained_versionr…
|
|||
|
|
s· € ð �!�}× Ñ Ô ØðܘJÓ'ˆð #€DÔÙ Ó ™$�BŠR˜$€EÐ ä
|
|||
|
|
ˆ5ƒz�Q‚˜5 ™8×,Ñ,° Ñ;Øä
|
|||
|
|
Ñ
|
|||
|
|
1©5Ó
|
|||
|
|
1Ô1ØÜÑ?¹Ó?Ó?€IÜÑ?¹Ó?Ó?€Iá‘YØà øô! ò Ùðüò
|
|||
|
|
s – B ¬ B#Â B Â B Ú requirementc óZ — t | « }|j t |j « fS )z@Split a requirement like spacy>=1.2.3 into ("spacy", ">=1.2.3").)r r{ r Ú specifier)r† Úreqs rZ Úsplit_requirementrŠ ( s$ € ä
|
|||
|
|
�kÓ
|
|||
|
|
"€CØ �H‰H”c˜#Ÿ-™-Ó(Ð )Ð)rY c óV — t | « j }d| › d|d › d|d dz › d�S )z_Generate a version range like >=1.2.3,<1.3.0 based on a given version
|
|||
|
|
(e.g. of spaCy).
|
|||
|
|
rs z,<r r> r) z.0)r Úrelease)rP rŒ s rZ Úget_minor_version_ranger� . s< € ô �gÓ ×&Ñ&€GØ
|
|||
|
|
�ˆy˜˜7 1™:˜, a¨°©
|
|||
|
|
°Q©Ð'7°rÐ :Ð:rY c ó‚ — t | « }|D ] }|j dv sŒ|j c S y# t $ r Y yw xY w)z>From a version range like >=1.2.3,<1.3.0 return the lower pin.)rs rl z~=N)r rv rP Ú Exception)ri Úspecsetrn s rZ Úget_model_lower_versionr‘ 6 sN € ð
|
|||
|
|
Ü ˜zÓ*ˆÛˆDØ�}‰}Ð 2Ò2Ø—|‘|Ò#ñ ð
|
|||
|
|
øô ò
|
|||
|
|
Ø Ø ð
|
|||
|
|
ús ‚ 2 ¡
|
|||
|
|
2 ¯2 ² >½>c ó, — t | « j S )zªCheck whether a version is a prerelease version.
|
|||
|
|
|
|||
|
|
version (str): The version, e.g. "3.0.0.dev1".
|
|||
|
|
RETURNS (bool): Whether the version is a prerelease version.
|
|||
|
|
)r Ú
|
|||
|
|
is_prerelease©rP s rZ Úis_prerelease_versionr• B s € ô �7Ó × )Ñ )Ð)rY c ó, — t | « j S )z©Generate the base version without any prerelease identifiers.
|
|||
|
|
|
|||
|
|
version (str): The version, e.g. "3.0.0.dev1".
|
|||
|
|
RETURNS (str): The base version, e.g. "3.0.0".
|
|||
|
|
)r Ú base_versionr” s rZ Úget_base_versionr˜ K s € ô �7Ó × (Ñ (Ð(rY c óz — t | « }|j › d|j › �S # t t f$ r Y yw xY w)zºGet the major + minor version (without patch or prerelease identifiers).
|
|||
|
|
|
|||
|
|
version (str): The version.
|
|||
|
|
RETURNS (str): The major + minor version or None if version is invalid.
|
|||
|
|
Nr> )r Ú TypeErrorr ÚmajorÚminor)rP Úvs rZ Úget_minor_versionrž T sE € ð Ü �GÓ ˆð �g‰gˆY�a˜Ÿ™�yÐ !Ð!øô
|
|||
|
|
”~Ð &ò Ùðús ‚ ( ¨:¹:Ú version_aÚ version_bc óP — t | « }t |« }|duxr |duxr ||k( S )aN Compare two versions and check if they match in major and minor, without
|
|||
|
|
patch or prerelease identifiers. Used internally for compatibility checks
|
|||
|
|
that should be insensitive to patch releases.
|
|||
|
|
|
|||
|
|
version_a (str): The first version
|
|||
|
|
version_b (str): The second version.
|
|||
|
|
RETURNS (bool): Whether the versions match.
|
|||
|
|
N)rž )rŸ r ÚaÚbs rZ Úis_minor_version_matchr¤ a s4 € ô ˜)Ó$€Aܘ)Ó$€AØ �Dˆ=Ò 5˜Q d˜]Ò 5¨q°A©vÐ5rY c ó" — t | « } | j j « s3t t j
|
|||
|
|
j
|
|||
|
|
| j ¬« « ‚| j « r| j « s4t t j j
|
|||
|
|
| j d¬« « ‚t j | « }dD ]4 }||vs|| rŒ
|
|||
|
|
t t j j
|
|||
|
|
|¬« « ‚ d|v �rt t j |d « s„t! |d « }t# |« }|�d|z }nd|v r d |d z }nd
|
|||
|
|
}t$ j& j
|
|||
|
|
|d › d |d
|
|||
|
|
› �|d |t j ¬« }t) j* |« t- |d « r`t$ j. j
|
|||
|
|
|d › d |d
|
|||
|
|
› �|d |d t1 t j « ¬« }t) j* |« |S )z¢Load a model meta.json from a path and validate its contents.
|
|||
|
|
|
|||
|
|
path (Union[str, Path]): Path to meta.json.
|
|||
|
|
RETURNS (Dict[str, Any]): The loaded meta.
|
|||
|
|
rö ú meta.jsonrY )rÒ r{ rP )ÚsettingÚ
|
|||
|
|
spacy_versionr� Úspacy_git_versionz git commit zversion unknownrÒ rr r{ rP )ÚmodelÚ
|
|||
|
|
model_versionrP Úcurrent)rª r« rP Úexample)r÷ r
|
|||
|
|
rù r r0 r& rŒ r^ r_ rú rû rÿ ÚE054ro r* Ú __version__r‘ r˜ r1 ÚW095ÚwarningsÚwarnr… ÚW094r� )rô r r§ Ú
|
|||
|
|
lower_versionÚwarn_msgs rZ Ú load_metar¶ o s× € ô �tÓ €DØ �;‰;× Ñ Ô Ü”f—k‘k×(Ñ(¨d¯k©kÐ(Ó:Ó;Ð;Ø �;‰;Œ= § ¡ ¤Ü”f—k‘k×(Ñ(¨d¯k©kÀ Ð(ÓLÓMÐMÜ �?‰?˜4Ó €DÛ.ˆØ ˜$Ñ d¨7£mÜ œVŸ[™[× /Ñ /¸Ð /Ó @ÓAÐ Að /ð ˜$Ò Ü$¤U×%6Ñ%6¸¸_Ñ8MÔNÜ 3°D¸ Ñ4IÓ JˆMÜ ,¨]Ó ;ˆMØ Ð(Ø # mÑ 3‘
|
|||
|
|
Ø$¨Ñ,Ø -°Ð5HÑ0IÑ I‘
|
|||
|
|
à 1�
|
|||
|
|
Ü —}‘}×+Ñ+Ø ˜f™ ˜ a¨¨V© ~Ð6Ø " 9™oØ%Ü ×)Ñ)ð ,ó ˆHô
|
|||
|
|
�M‰M˜(Ô #Ü # D¨ Ñ$9Ô :Ü —}‘}×+Ñ+Ø ˜f™ ˜ a¨¨V© ~Ð6Ø " 9™oØ ˜_Ñ-Ü/´×0AÑ0AÓBð ,ó ˆHô
|
|||
|
|
�M‰M˜(Ô #Ø €KrY c ó4 — t | « }t |dz « S )z¯Get model meta.json from a directory path and validate its contents.
|
|||
|
|
|
|||
|
|
path (str / Path): Path to model directory.
|
|||
|
|
RETURNS (Dict[str, Any]): The model's meta data.
|
|||
|
|
r¦ )r÷ r¶ )rô r! s rZ r' r' š s € ô ˜TÓ"€JÜ �Z +Ñ-Ó .Ð.rY c ó> — t j | « y# Y yxY w)z˜Check if string maps to a package installed via pip.
|
|||
|
|
|
|||
|
|
name (str): Name of package.
|
|||
|
|
RETURNS (bool): True if installed package, False if not.
|
|||
|
|
TF)r- Ú distributionr s rZ r r ¤ s" € ð Ü×'Ñ'¨Ô-ØøðÙús ‚ ˜ c ó´ — t j | « }t t t t
|
|||
|
|
t j f |j « « j S )zxGet the path to an installed package.
|
|||
|
|
|
|||
|
|
name (str): Package name.
|
|||
|
|
RETURNS (Path): Path to installed package.
|
|||
|
|
)
|
|||
|
|
rè ré r r r r r r r r
|
|||
|
|
)r{ Úpkgs rZ Úget_package_pathr¼ ± s> € ô ×
|
|||
|
|
!Ñ
|
|||
|
|
! $Ó
|
|||
|
|
'€CÜ ””Uœ3¤§ ¡ Ð+Ñ,¨c¯l©lÓ;Ó <× CÑ CÐCrY rª ÚtargetÚ replacementc ó4 — | j « D ]9 }||j v sŒ||j |j j |« < Œ; | j « D ]9 }|j D ]( }|j |« |u sŒ|j ||« Œ* Œ; y)zÊReplace a node within a model with a new one, updating refs.
|
|||
|
|
|
|||
|
|
model (Model): The parent model.
|
|||
|
|
target (Model): The target node.
|
|||
|
|
replacement (Model): The node to replace the target with.
|
|||
|
|
N)ÚwalkÚlayersÚindexÚ ref_namesÚ
|
|||
|
|
maybe_get_refÚset_ref)rª r½ r¾ ÚnodeÚref_names rZ Úreplace_model_noderÈ ½ s~ € ð —
|
|||
|
|
‘
|
|||
|
|
– ˆØ �T—[‘[Ò Ø5@ˆD�K‰K˜Ÿ ™ ×)Ñ)¨&Ó1Ò 2ð ð —
|
|||
|
|
‘
|
|||
|
|
– ˆØ ŸœˆHØ×!Ñ! (Ó+¨vÒ5Ø— ‘ ˜X {Õ3ñ 'ñ rY Úcommandc ó: — t j | t ¬« S )z�Split a string command using shlex. Handles platform compatibility.
|
|||
|
|
command (str) : The command to split
|
|||
|
|
RETURNS (List[str]): The split command.
|
|||
|
|
)Úposix)ÚshlexrB r. )rÉ s rZ Ú
|
|||
|
|
split_commandrÍ Ï s € ô
|
|||
|
|
�;‰;�w¬* nÔ 5Ð5rY )r\ Úcapturer\ rÎ c óÐ — t | t « rt | « }| }n| }dj | « } t j
|
|||
|
|
|t j j « |dd|rt j nd|rt j nd¬« }|j dk7 r_|r]d|› d �}|d
|
|||
|
|
|j › �z
|
|||
|
|
}|j �|d z
|
|||
|
|
}||j z
|
|||
|
|
}t j" |« }||_ ||_ |‚|j dk7 r t) j* |j « |S # t $ r/ t t j j ||d ¬« « d‚w xY w) aì Run a command on the command line as a subprocess. If the subprocess
|
|||
|
|
returns a non-zero exit code, a system exit is performed.
|
|||
|
|
command (str / List[str]): The command. If provided as a string, the
|
|||
|
|
string will be split using shlex.split.
|
|||
|
|
stdin (Optional[Any]): stdin to read from or None.
|
|||
|
|
capture (bool): Whether to capture the output and errors. If False,
|
|||
|
|
the stdout and stderr will not be redirected, and if there's an error,
|
|||
|
|
sys.exit will be called with the return code. You should use capture=False
|
|||
|
|
when you want to turn over execution to the command, and capture=True
|
|||
|
|
when you want to run the command more like a function.
|
|||
|
|
RETURNS (Optional[CompletedProcess]): The process object.
|
|||
|
|
ú Úutf8FN)ÚenvÚinputÚencodingÚcheckÚstdoutÚstderrr )Ú str_commandÚtoolzError running command:
|
|||
|
|
|
|||
|
|
z
|
|||
|
|
|
|||
|
|
z Subprocess exited with status z$
|
|||
|
|
|
|||
|
|
Process log (stdout and stderr):
|
|||
|
|
|
|||
|
|
)rv r rÍ rŠ Ú
|
|||
|
|
subprocessÚrunr ÚenvironÚcopyÚPIPEÚSTDOUTÚFileNotFoundErrorr0 ÚE970rŒ Ú
|
|||
|
|
returncoderÖ ÚSubprocessErrorÚretrÉ r Úexit)rÉ r\ rÎ Úcmd_listÚcmd_strrä Úmessager¦ s rZ Ú run_commandré × sO € ô$ �'œ3Ô Ü Ó)ˆØ‰àˆØ—(‘(˜7Ó#ˆðÜ�n‰nØ Ü—
|
|||
|
|
‘
|
|||
|
|
—‘Ó!ØØØÙ&-”:—?’?°4Ù(/”:×$Ò$°Tô
|
|||
|
|
ˆð ‡~�~˜Ò™wØ.¨w¨i°tÐ<ˆØÐ3°C·N±NÐ3CÐDÑDˆØ �:‰:Ð !Ø ÐAÑ BˆGØ �s—z‘zÑ !ˆGÜ×*Ñ*¨7Ó3ˆØˆŒ Ø ˆŒ
|
|||
|
|
؈ Ø � ‰ ˜1Ò Ü �‰�— ‘ Ô Ø €Jøô% ò ô Ü �K‰K× Ñ ¨7¸À!¹Ð Ó Eó
|
|||
|
|
àð ðús ³A D- Ä-8E%c # ó. K — t j « }t | « j « }t j t |« « |–— t j t |« « y# t j t |« « w xY ww)aU Change current working directory and returns to previous on exit.
|
|||
|
|
path (str / Path): The directory to navigate to.
|
|||
|
|
YIELDS (Path): The absolute path to the current working directory. This
|
|||
|
|
should be used if the block needs to perform actions within the working
|
|||
|
|
directory, to prevent mismatches with relative paths.
|
|||
|
|
N)r ÚcwdrD r Úchdirr )rô Úprev_cwdr¬ s rZ Ú working_dirrî s_ è ø€ ô �x‰x‹z€HÜ�4‹j× Ñ Ó"€G܇H�HŒS�‹\Ôð ØŠ
|
|||
|
|
ä
|
|||
|
|
�‰”�X“ Õ øŒ�‰”�X“ Õ üs ‚A BÁA2 Á BÁ2 BÂBc # ó’ K — t t j « « } | –— d„ } t j dk\ r!t j t | « |¬« yt j t | « |¬« y# t $ r>}t j t j j | |¬« « Y d}~yd}~ww xY ww)z¬Execute a block in a temporary directory and remove the directory and
|
|||
|
|
its contents at the end of the with block.
|
|||
|
|
YIELDS (Path): The path of the temp directory.
|
|||
|
|
c ó\ — t j |t j « | |« y r¯ )r ÚchmodÚstatÚS_IWRITE)Úrmfuncrô Úexs rZ Ú force_removez"make_tempdir.<locals>.force_remove* s € Ü
|
|||
|
|
�‰�”t—}‘}Ô%Ùˆt� rY )é é )Úonexc)Úonerror)Údirr@ N)r ÚtempfileÚmkdtempr Ú version_infoÚshutilÚrmtreer ÚPermissionErrorr± r² r1 ÚW091rŒ )Údrö Úes rZ Ú make_tempdirr sŽ è ø€ ô
|
|||
|
|
ŒX×
|
|||
|
|
Ñ
|
|||
|
|
Ó
|
|||
|
|
Ó €AØ
|
|||
|
|
‚Gòð:Ü × Ñ ˜wÒ &Ü �M‰Mœ#˜a›&¨ Ö 5ä �M‰Mœ#˜a›&¨,Ö 7øÜ ò :Ü�
|
|||
|
|
‰
|
|||
|
|
”h—m‘m×*Ñ*¨q°aÐ*Ó8×9Ñ9ûð:üs: ‚%C¨3A= ÁCÁ A= Á<CÁ= CÂ4B?Â:CÂ?CÃCc óÖ — t « j j dk( ryt « j j dk( ry ddl} y# t $ r Y Œw xY w# t $ r Y yw xY w)zÎCheck if user is running spaCy from a Jupyter or Colab notebook by
|
|||
|
|
detecting the IPython kernel. Mainly used for the displaCy visualizer.
|
|||
|
|
RETURNS (bool): True if in Jupyter/Colab, False if not.
|
|||
|
|
ÚZMQInteractiveShellTzgoogle.colab._shellr NF)Ú get_ipythonr rT rU Ú NameErrorÚ google.colabrê )Úgoogles rZ Ú
|
|||
|
|
is_in_jupyterr 7 sv € ð
|
|||
|
|
Ü ‹=× "Ñ "× +Ñ +Ð/DÒ DØÜ ‹=× "Ñ "× -Ñ -Ð1FÒ FØð Gð
|
|||
|
|
|
|||
|
|
Ûàøô
|
|||
|
|
ò
|
|||
|
|
Ù ð
|
|||
|
|
ûô ò
|
|||
|
|
Ø Ø ð
|
|||
|
|
ús( ‚!A
|
|||
|
|
¤!A
|
|||
|
|
ÁA Á
|
|||
|
|
AÁAÁ A(Á'A(c óF — t t d« xs t t d« S )z³Check if user is running spaCy from an interactive Python
|
|||
|
|
shell. Will return True in Jupyter notebooks too.
|
|||
|
|
RETURNS (bool): True if in interactive mode, False if not.
|
|||
|
|
Úps1Úps2)r‰ r rX rY rZ Úis_in_interactiver O s € ô ”3˜Ó Ò 5¤'¬#¨uÓ"5Ð5rY Úobjc ó — t | d« r| j � | j S t | d« r | j S t | d« r,t | j d« r| j j S t | « S )z¹Get a human-readable name of a Python object, e.g. a pipeline component.
|
|||
|
|
|
|||
|
|
obj (Any): The Python object, typically a function or class.
|
|||
|
|
RETURNS (str): A human-readable name.
|
|||
|
|
r{ rT r )r‰ r{ rT r r )r s rZ Úget_object_namer X se € ô ˆs�FÔ §¡Ð 4Ø�x‰xˆÜˆs�JÔ Ø�|‰|Ð܈s�KÔ ¤W¨S¯]©]¸JÔ%GØ�}‰}×%Ñ%Ð%Ü �‹9ÐrY Úfunc1Úfunc2c óT — t | « r t |« syt | d« r t |d« sy| j |j k( }t j | « t j |« k( }t j
|
|||
|
|
| « t j
|
|||
|
|
|« k( }|xr |xr |S )aó Approximately decide whether two functions are the same, even if their
|
|||
|
|
identity is different (e.g. after they have been live reloaded). Mostly
|
|||
|
|
used in the @Language.component and @Language.factory decorators to decide
|
|||
|
|
whether to raise if a factory already exists. Allows decorator to run
|
|||
|
|
multiple times with the same function.
|
|||
|
|
|
|||
|
|
func1 (Callable): The first function.
|
|||
|
|
func2 (Callable): The second function.
|
|||
|
|
RETURNS (bool): Whether it's the same function (most likely).
|
|||
|
|
FrV )Úcallabler‰ rV rs ÚgetfileÚgetsourcelines)r r Ú same_nameÚ same_fileÚ same_codes rZ Ú is_same_funcr g sŒ € ô �EŒ?¤(¨5¤/ØÜ �5˜.Ô )´¸ÀÔ1OØØ×"Ñ" e×&8Ñ&8Ñ8€IÜ—‘ Ó&¬'¯/©/¸%Ó*@Ñ@€IÜ×&Ñ& uÓ-´×1GÑ1GÈÓ1NÑN€IØ Ò 0˜Ò 0 yÐ0rY ÚrequireÚ non_blockingc ó^ — t « }t €y t |t « ry t |¬« S )N)r )r% r+ rv r# )r r Úopss rZ Úget_cuda_streamr" | s- € ô Ó
|
|||
|
|
€CÜÐØÜ �CœÔ "Øä |Ô4Ð4rY c ó” — t €|S t j |j d|j ¬« }|j || ¬« |S )NÚC)ÚorderÚdtype)Ústream)r, ÚndarrayÚshaper& Úset)r' Ú numpy_arrayÚarrays rZ Ú get_asyncr- ˆ s@ € Ü €|ØÐä— ‘ ˜[× .Ñ .°cÀ×ARÑARÔSˆØ
|
|||
|
|
� ‰ �+ fˆ Ô-؈ rY c ód — t | « } | j d¬« 5 }|j « j d« }d d d « dj D �cg c]+ }|j « sŒdt
|
|||
|
|
j |« z ‘Œ- c}« }t
|
|||
|
|
j |« S # 1 sw Y ŒdxY wc c}w )NrÑ )rÔ ú
|
|||
|
|
ú|ú^) r÷ Úopenr] rB rŠ ÚstripÚreÚescapeÚcompile)rô Úfile_ÚentriesÚpieceÚ
|
|||
|
|
expressions rZ Ú
|
|||
|
|
read_regexr; ‘ sŒ € Ü �tÓ €DØ
|
|||
|
|
�‰˜FˆÔ # uØ—*‘*“,×$Ñ$ TÓ*ˆ÷
|
|||
|
|
$à—‘Ù-4ÓF©W E¸¿ ¹ ½
|
|||
|
|
ˆŒr�y‰y˜Ó Ó ¨WÑFó€Jô �:‰:�jÓ !Ð!÷
|
|||
|
|
$Ð #üò Gs ž B!ÁB-Á+B-Â!B*r8 c óœ — dj | D �cg c] }|j « sŒd|z ‘Œ c}« }t j |« S c c}w )a Compile a sequence of prefix rules into a regex object.
|
|||
|
|
|
|||
|
|
entries (Iterable[Union[str, Pattern]]): The prefix rules, e.g.
|
|||
|
|
spacy.lang.punctuation.TOKENIZER_PREFIXES.
|
|||
|
|
RETURNS (Pattern): The regex object. to be used for Tokenizer.prefix_search.
|
|||
|
|
r0 r1 ©rŠ r3 r4 r6 ©r8 r9 r: s rZ Úcompile_prefix_regexr? › sB € ð —‘±GÓM±G¨5¸u¿{¹{½}˜3 ›;°GÑMÓN€JÜ
|
|||
|
|
�:‰:�jÓ !Ð!ùò Nó
|
|||
|
|
�A ¦A c óœ — dj | D �cg c] }|j « sŒ|dz ‘Œ c}« }t j |« S c c}w )a Compile a sequence of suffix rules into a regex object.
|
|||
|
|
|
|||
|
|
entries (Iterable[Union[str, Pattern]]): The suffix rules, e.g.
|
|||
|
|
spacy.lang.punctuation.TOKENIZER_SUFFIXES.
|
|||
|
|
RETURNS (Pattern): The regex object. to be used for Tokenizer.suffix_search.
|
|||
|
|
r0 ú$r= r> s rZ Úcompile_suffix_regexrC ¦ sB € ð —‘±GÓM±G¨5¸u¿{¹{½}˜5 3›;°GÑMÓN€JÜ
|
|||
|
|
�:‰:�jÓ !Ð!ùò Nr@ c ó– — dj | D �cg c] }|j « sŒ|‘Œ c}« }t j |« S c c}w )a Compile a sequence of infix rules into a regex object.
|
|||
|
|
|
|||
|
|
entries (Iterable[Union[str, Pattern]]): The infix rules, e.g.
|
|||
|
|
spacy.lang.punctuation.TOKENIZER_INFIXES.
|
|||
|
|
RETURNS (regex object): The regex object. to be used for Tokenizer.infix_finditer.
|
|||
|
|
r0 r= r> s rZ Úcompile_infix_regexrE ± s> € ð —‘©gÓG©g U¸¿¹½ š5¨gÑGÓH€JÜ
|
|||
|
|
�:‰:�jÓ !Ð!ùò Hs
|
|||
|
|
�A¦AÚ default_funcc ó8 — t j t | |« S )aQ Extend an attribute function with special cases. If a word is in the
|
|||
|
|
lookups, the value is returned. Otherwise the previous function is used.
|
|||
|
|
|
|||
|
|
default_func (callable): The default function to execute.
|
|||
|
|
*lookups (dict): Lookup dictionary mapping string to attribute value.
|
|||
|
|
RETURNS (callable): Lexical attribute getter.
|
|||
|
|
)Ú functoolsÚpartialÚ_get_attr_unless_lookup)rF rb s rZ Ú add_lookupsrK ¼ s € ô × Ñ Ô 4°lÀGÓ LÐLrY rb Ústringc ó6 — |D ]
|
|||
|
|
}||v sŒ|| c S | |« S r¯ rX )rF rb rL Úlookups rZ rJ rJ É s- € ó ˆØ �VÒ Ø˜&‘>Ò !ð ñ ˜Ó Ð rY Úbase_exceptionsc óœ — t | « }|D ]® }|j « D ]ˆ \ }}t d„ |D « « s*t t j
|
|||
|
|
j
|
|||
|
|
||¬« « ‚dj d„ |D « « }||k7 sŒ`t t j j
|
|||
|
|
||¬« « ‚ |j |« Œ° t |dd« }|S )a' Update and validate tokenizer exceptions. Will overwrite exceptions.
|
|||
|
|
|
|||
|
|
base_exceptions (Dict[str, List[dict]]): Base exceptions.
|
|||
|
|
*addition_dicts (Dict[str, List[dict]]): Exceptions to add to the base dict, in order.
|
|||
|
|
RETURNS (Dict[str, List[dict]]): Combined tokenizer exceptions.
|
|||
|
|
c 3 óJ K — | ] }t |t t « –— Œ y wr¯ )rv r3 r ©rx Úattrs rZ rz z update_exc.<locals>.<genexpr>Þ s è ø€ ÐK¹{°t”z $¤t¡*¬c×2¹{ùs ‚!#)r² Úorthsr c 3 ó. K — | ]
|
|||
|
|
}|t –— Œ y wr¯ r2 rR s rZ rz z update_exc.<locals>.<genexpr>à s è ø€ Ð$H¹K°D T¬$¥Z¹Kùó ‚ú'u ’) Údictr9 Úallrÿ r0 ÚE055rŒ rŠ ÚE056rº Ú
|
|||
|
|
expand_exc)rO Úaddition_dictsÚexcÚ additionsÚorthÚ token_attrsÚdescribed_orths rZ Ú
|
|||
|
|
update_excrc Ò s¹ € ô ˆÓ
|
|||
|
|
€CÛ#ˆ Ø!*§ ¡ Ö!2Ñ ˆD�+ÜÑK¹{ÓKÔKÜ ¤§¡×!3Ñ!3¸ÀKÐ!3Ó!PÓQÐQØ ŸW™WÑ$H¹KÓ$HÓ HˆNØ�~Ó%Ü ¤§¡×!3Ñ!3¸ÀNÐ!3Ó!SÓTÐTð "3ð �
|
|||
|
|
‰
|
|||
|
|
�9Õ ð $ô �S˜#˜uÓ
|
|||
|
|
%€CØ €JrY ÚexcsÚsearchr� c ó — d„ }t | « }| j « D ]8 \ }}||v sŒ |j ||« }|D �cg c] } ||||« ‘Œ } }| ||< Œ: |S c c}w )ab Find string in tokenizer exceptions, duplicate entry and replace string.
|
|||
|
|
For example, to add additional versions with typographic apostrophes.
|
|||
|
|
|
|||
|
|
excs (Dict[str, List[dict]]): Tokenizer exceptions.
|
|||
|
|
search (str): String to find and replace.
|
|||
|
|
replace (str): Replacement.
|
|||
|
|
RETURNS (Dict[str, List[dict]]): Combined tokenizer exceptions.
|
|||
|
|
c ó\ — t | « }|t j ||« |t < |S r¯ )rX r3 r� )Útokenre r� Úfixeds rZ Ú
|
|||
|
|
_fix_tokenz expand_exc.<locals>._fix_tokenô s* € Ü�U“ ˆØœD‘k×)Ñ)¨&°'Ó:ˆŒd‰ ؈ rY )rX r9 r� )
|
|||
|
|
rd re r� rj Únew_excsÚ token_stringÚtokensÚnew_keyÚtÚ new_values
|
|||
|
|
rZ r\ r\ è su € òô
|
|||
|
|
�D‹z€HØ $§
|
|||
|
|
¡
|
|||
|
|
¦ Ñ ˆ �fØ �\Ò !Ø"×*Ñ*¨6°7Ó;ˆGÙAGÓHÁ¸A™ A v¨wÕ7ÀˆIÐHØ )ˆH�WÒ ð !-ð
|
|||
|
|
€Oùò Is Á A ÚlengthÚstartÚstopÚstepc óÞ — |� |dk( st t j « ‚|€d}n
|
|||
|
|
|dk r|| z
|
|||
|
|
}t | t d|« « }|€| }n
|
|||
|
|
|dk r|| z
|
|||
|
|
}t | t ||« « }||fS )Nr) r )rÿ r0 ÚE057ÚminÚmax)rq rr rs rt s rZ Únormalize_slicery s� € ð
|
|||
|
|
ˆL˜D AšIÜœŸ™Ó%Ð%Ø €}Ø‰Ø �ŠØ
|
|||
|
|
�‰ˆÜ �œ˜A˜u›
|
|||
|
|
Ó &€EØ €|؉Ø
|
|||
|
|
�ŠØ �‰ˆÜ ˆv”s˜5 $Ó'Ó (€DØ �$ˆ;ÐrY Úspansr7 c ó, — d„ }t | |d¬« }g }t « }|D ]c }|j |vsŒ|j dz
|
|||
|
|
|vsŒ$|j |« |j t
|
|||
|
|
|j |j « « Œe t |d„ ¬« }|S )a€ Filter a sequence of spans and remove duplicates or overlaps. Useful for
|
|||
|
|
creating named entities (where one token can only be part of one entity) or
|
|||
|
|
when merging spans with `Retokenizer.merge`. When spans overlap, the (first)
|
|||
|
|
longest span is preferred over shorter spans.
|
|||
|
|
|
|||
|
|
spans (Iterable[Span]): The spans to filter.
|
|||
|
|
RETURNS (List[Span]): The filtered spans.
|
|||
|
|
c óN — | j | j z
|
|||
|
|
| j fS r¯ )Úendrr ©Úspans rZ ú<lambda>z filter_spans.<locals>.<lambda> s € §¡¨D¯J©JÑ!6¸¿¹¸ Ñ DrY T)r² rÎ r) c ó — | j S r¯ )rr r~ s rZ r€ z filter_spans.<locals>.<lambda>& s € ¨T¯ZªZrY )r² )rx r* rr r} rw rº Úrange)rz Ú get_sort_keyÚ sorted_spansrK Ú seen_tokensr s rZ Ú filter_spansr† s† € ñ E€Lܘ% \¸4Ô@€LØ
|
|||
|
|
€FÜ ›E€KÛ ˆà �:‰:˜[Ò (¨T¯X©X¸©\ÀÒ-LØ �M‰M˜$Ô Ø × Ñ œu T§Z¡Z°·±Ó :Õ ;ð ô
|
|||
|
|
�FÑ 7Ô
|
|||
|
|
8€FØ €MrY c ó8 — t t j | Ž « S r¯ )r† Ú itertoolsÚchain)rz s rZ Úfilter_chain_spansrŠ * s € Ü œ Ÿ™¨Ð/Ó 0Ð0rY z#spacy.first_longest_spans_filter.v1c ó — t S r¯ )rŠ rX rY rZ Ú make_first_longest_spans_filterrŒ . s € ä Ð rY Úgettersc ó@ — t j t | |« « S r¯ )rú Ú
|
|||
|
|
msgpack_dumpsÚto_dict)r� r s rZ Úto_bytesr‘ 3 s € Ü × Ñ œw w°Ó 8Ó 9Ð9rY Ú
|
|||
|
|
bytes_dataÚsettersc óB — t t j | « ||« S r¯ )Ú from_dictrú Ú
|
|||
|
|
msgpack_loads)r’ r“ r s rZ Ú
|
|||
|
|
from_bytesr— 7 s € ô
|
|||
|
|
”U×(Ñ(¨Ó4°g¸wÓ GÐGrY c ó| — i }| j « D ]& \ }}|j d« d |vsŒ |« ||< Œ( |S ©Nr> r ©r9 rB )r� r Ú
|
|||
|
|
serializedr² Úgetters rZ r� r� ? sE € ð €JØ —}‘}–‰ ˆˆVà �9‰9�S‹>˜!Ñ GÒ +Ù $›hˆJ�sŠOð 'ð ÐrY r@ c ó„ — |j « D ], \ }}|j d« d |vsŒ || v sŒ" || | « Œ. | S r™ rš )r@ r“ r r² Úsetters rZ r• r• J sF € ð
|
|||
|
|
—}‘}–‰ ˆˆVà �9‰9�S‹>˜!Ñ GÒ +°°s²
|
|||
|
|
Ù �3�s‘8Õ ð 'ð €JrY Úwritersc óÐ — t | « } | j « s| j « |j « D ]' \ }}|j d« d |vsŒ || |z « Œ) | S r™ )r÷ rù Úmkdirr9 rB )rô rŸ r r² Úwriters rZ Úto_diskr£ V s\ € ô
|
|||
|
|
�tÓ €DØ �;‰;Œ=Ø �
|
|||
|
|
‰
|
|||
|
|
Œ Ø —}‘}–‰ ˆˆVà �9‰9�S‹>˜!Ñ GÒ +Ù �4˜#‘:Õ ð 'ð €KrY rg c ó� — t | « } |j « D ]' \ }}|j d« d |vsŒ || |z « Œ) | S r™ )r÷ r9 rB )rô rg r r² Úreaders rZ r+ r+ e sJ € ô
|
|||
|
|
�tÓ €DØ —}‘}–‰ ˆˆVà �9‰9�S‹>˜!Ñ GÒ +Ù �4˜#‘:Õ ð 'ð €KrY r? c óÌ — t j j | t |« « }t j j |« }|j
|
|||
|
|
j
|
|||
|
|
|« |S )z¶Import module from a file. Used to load models from a directory.
|
|||
|
|
|
|||
|
|
name (str): Name of module to load.
|
|||
|
|
loc (str / Path): Path to the file.
|
|||
|
|
RETURNS: The loaded module.
|
|||
|
|
)rè ÚutilÚspec_from_file_locationr Úmodule_from_specÚloaderÚ exec_module)r{ r? rn rð s rZ Ú import_filer¬ r sI € ô �>‰>× 1Ñ 1°$¼¸C»Ó A€DÜ
|
|||
|
|
�^‰^×
|
|||
|
|
,Ñ
|
|||
|
|
,¨TÓ
|
|||
|
|
2€F؇K�K×јFÔ#Ø €MrY Úhtmlc ób — | j « j dd« j dd« S )zïPerform a template-specific, rudimentary HTML minification for displaCy.
|
|||
|
|
Disclaimer: NOT a general-purpose solution, only removes indentation and
|
|||
|
|
newlines.
|
|||
|
|
|
|||
|
|
html (str): Markup to minify.
|
|||
|
|
RETURNS (str): "Minified" HTML.
|
|||
|
|
z r r/ )r3 r� )r s rZ Ú minify_htmlr¯ s* € ð �:‰:‹<× Ñ ¨Ó +× 3Ñ 3°D¸"Ó =Ð=rY c ó– — | j dd« } | j dd« } | j dd« } | j dd« } | S ) zæReplace <, >, &, " with their HTML encoded representation. Intended to
|
|||
|
|
prevent HTML errors in rendered displaCy markup.
|
|||
|
|
|
|||
|
|
text (str): The original text.
|
|||
|
|
RETURNS (str): Equivalent text to be safely used within HTML.
|
|||
|
|
ú&z&r} z<rr z>ú"z")r� )r` s rZ Ú escape_htmlr³ Š sI € ð �<‰<˜˜WÓ %€DØ �<‰<˜˜VÓ $€DØ �<‰<˜˜VÓ $€DØ �<‰<˜˜XÓ &€DØ €KrY Úwordsc óf — dj dj | « j « « dj |j « « k7 r*t t j j || ¬« « ‚g }g }d}| D �cg c] }|j
|
|||
|
|
« rŒ|‘Œ }}|D ]› } ||d j |« }|dkD r-|j ||||z « |j d« ||z
|
|||
|
|
}|j |« |j d« |t |« z
|
|||
|
|
}|t |« k sŒ‰|| dk( sŒ’d|d< |d z
|
|||
|
|
}Œ� |t |« k r%|j ||d « |j d« ||fS c c}w # t $ r, t t j j || ¬« « d‚w xY w)
|
|||
|
|
a† Given a list of words and a text, reconstruct the original tokens and
|
|||
|
|
return a list of words and spaces that can be used to create a Doc. This
|
|||
|
|
can help recover destructive tokenization that didn't preserve any
|
|||
|
|
whitespace information.
|
|||
|
|
|
|||
|
|
words (Iterable[str]): The words.
|
|||
|
|
text (str): The original text.
|
|||
|
|
RETURNS (Tuple[List[str], List[bool]]): The words and spaces.
|
|||
|
|
r )r` r´ r NFrÐ Téÿÿÿÿr) )
|
|||
|
|
rŠ rB rÿ r0 ÚE194rŒ Úisspacer rw r€ )r´ r` Ú
|
|||
|
|
text_wordsÚ text_spacesÚtext_posÚwordÚ
|
|||
|
|
norm_wordsÚ
|
|||
|
|
word_starts rZ Úget_words_and_spacesr¿ ˜ s¨ € ð
|
|||
|
|
‡w�wˆr�w‰w�u‹~×#Ñ#Ó%Ó&¨"¯'©'°$·*±*³,Ó*?Ò?ÜœŸ™×+Ñ+°¸UÐ+ÓCÓDÐDØ€JØ€KØ€Há#(Ó?¡5˜4°· ± µ’$ 5€JÐ?ãˆð SØ ˜h˜i˜ ×.Ñ.¨tÓ4ˆJ𠘊>Ø × Ñ ˜d 8¨h¸Ñ.CÐ DÔ EØ × Ñ ˜uÔ %Ø ˜
|
|||
|
|
Ñ "ˆHØ×ј$Ô Ø×ј5Ô!Ø”C˜“IÑ ˆØ ”c˜$“iÓ D¨¡N°cÓ$9Ø "ˆK˜‰OØ ˜‰M‰Hð ð ”#�d“)ÒØ×ј$˜x˜y˜/Ô*Ø×ј5Ô!Ø ˜ Ð $Ð$ùò) @øô
|
|||
|
|
ò SÜ œVŸ[™[× /Ñ /°TÀÐ /Ó GÓHÈdÐ Rð Sús ÂE6ÂE6Â%E;Å;5F0c ó¤ — t | « j « S # t $ r+ t t j j | ¬« « d‚w xY w)zµDeep copy a Config. Will raise an error if the config contents are not
|
|||
|
|
JSON-serializable.
|
|||
|
|
|
|||
|
|
config (Config): The config to copy.
|
|||
|
|
RETURNS (Config): The copied config.
|
|||
|
|
r0 N)r rÝ rÿ r0 ÚE961rŒ r0 s rZ Ú copy_configr Á sK € ðFÜ�f‹~×"Ñ"Ó$Ð$øÜ ò FÜœŸ™×+Ñ+°6Ð+Ó:Ó;ÀÐEðFús ‚ ›4AÚvaluesc óð — i }| j « D ]` \ }}|}|j « j d« }t |« D ], \ }}|t |« dz
|
|||
|
|
k( }|j ||r|ni « }Œ. Œb |S )a Convert dot notation to a dict. For example: {"token.pos": True,
|
|||
|
|
"token._.xyz": True} becomes {"token": {"pos": True, "_": {"xyz": True }}}.
|
|||
|
|
|
|||
|
|
values (Dict[str, Any]): The key/value pairs to convert.
|
|||
|
|
RETURNS (Dict[str, dict]): The converted values.
|
|||
|
|
r> r) )r9 ÚlowerrB Ú enumerater€ Ú
|
|||
|
|
setdefault) rà rK r² r| rô ÚpartsÚiÚitemÚis_lasts rZ Ú dot_to_dictrÌ Î sx € ð !€FØ —l‘l–n‰
|
|||
|
|
ˆˆU؈ؗ ‘ “ ×!Ñ! #Ó&ˆÜ Ö'‰GˆAˆtØœ3˜u›:¨™>Ñ)ˆGØ—?‘? 4±'©¸rÓB‰Dñ (ð %ð €MrY r# r$ c ón — t | |¬« D ��ci c] \ }}dj |« |“Œ c}}S c c}}w )ae Convert dot notation to a dict. For example: {"token": {"pos": True,
|
|||
|
|
"_": {"xyz": True }}} becomes {"token.pos": True, "token._.xyz": True}.
|
|||
|
|
|
|||
|
|
obj (Dict[str, dict]): The dict to convert.
|
|||
|
|
for_overrides (bool): Whether to enable special handling for registered
|
|||
|
|
functions in overrides.
|
|||
|
|
RETURNS (Dict[str, Any]): The key/value pairs.
|
|||
|
|
r# r> )Ú walk_dictrŠ )r r$ r² r| s rZ r( r( ß sA € ô $ C°}ÕEô áE‰JˆC�ð �‰�‹
|
|||
|
|
�uÑ ØEò ð ùó s ‘ 1rJ c óÀ — | }|j d« }|D ] } || }Œ
|
|||
|
|
|S # t t f$ r+ t t j j |¬« « d‚w xY w)a` Convert dot notation of a "section" to a specific part of the Config.
|
|||
|
|
e.g. "training.optimizer" would return the Optimizer object.
|
|||
|
|
Throws an error if the section is not defined in this config.
|
|||
|
|
|
|||
|
|
config (Config): The config.
|
|||
|
|
section (str): The dot notation of the section in the config.
|
|||
|
|
RETURNS: The object denoted by the section
|
|||
|
|
r> r N)rB rF rš r0 ÚE952rŒ )r rJ Ú componentrÈ rÊ s rZ rE rE î so € ð €IØ �M‰M˜#Ó €EÛˆð GØ! $™‰Ið ð
|
|||
|
|
Ðøô œ)Ð$ò GÜœ6Ÿ;™;×-Ñ-°7Ð-Ó;Ó<À$Ð Fð Gús š#£:A r| c ó — | }|j d« }t |« D ]" \ }} |t |« dz
|
|||
|
|
k( r|||< n|| }Œ$ y# t t f$ r+ t t
|
|||
|
|
j j |¬« « d‚w xY w)zÒUpdate a config at a given position from a dot notation.
|
|||
|
|
|
|||
|
|
config (Config): The config.
|
|||
|
|
section (str): The dot notation of the section in the config.
|