117 lines
11 KiB
Plaintext
117 lines
11 KiB
Plaintext
|
Ë
|
|||
|
|
=û g…% ã ó^ — d dl Z d dlZd dlZd dlmZ d dlmZmZmZm Z m
|
|||
|
|
Z
|
|||
|
|
d dl Z d dl Z
|
|||
|
|
ddlmZ ddlmZ ddlmZmZ ddlmZ d d
|
|||
|
|
lmZmZmZmZm Z d d d
|
|||
|
|
dœZ ej< dd
|
|||
|
|
ddœ¬« edd¬« eddd¬« edd¬« edd¬« edd¬« ee d d d d ¬« edd d d!¬« ee d" d#d$d%¬« ee d& d'd(d)¬« ed
|
|||
|
|
d*d+d,d-¬« f
|
|||
|
|
d.e d/ed0e d1e d2e de d3e e d"e d&e!d4e!fd5„« Z"e d e d" e d& dd6œd.e d/ed0e d1e d2e de d"e d&e!d7e!d8e
|
|||
|
|
e#e#ee#e#f f fd9„Z$y):é N)ÚPath)ÚAnyÚDictÚListÚOptionalÚTupleé )Úutil)ÚErrors)ÚMultiLabel_TextCategorizerÚTextCategorizer)ÚCorpusé )ÚArgÚOptÚappÚ import_codeÚ setup_gpué éÿÿÿÿF)Ún_trialsÚuse_gpuÚ gold_preproczfind-thresholdT)Úallow_extra_argsÚignore_unknown_options)Úcontext_settings.zModel name or path)Úhelpz3Location of binary evaluation data in .spacy format)r Úexistsz&Name of pipe to examine thresholds forz7Key of threshold attribute in component's configurationzMetric to optimizer z
|
|||
|
|
--n_trialsz-nz0Number of trials to determine optimal thresholdsz--codez-czNPath to Python file with additional code (registered functions) to be importedr z--gpu-idz-gzGPU ID or -1 for CPUr z--gold-preprocz-GzUse gold preprocessingz --verbosez-Vz-VVz/Display more information for debugging purposesÚmodelÚ data_pathÚ pipe_nameÚ
|
|||
|
|
threshold_keyÚ
|
|||
|
|
scores_keyÚ code_pathÚverbosec
|
|||
|
|
ó — | r-t j j t j « t |« t
|
|||
|
|
| |||||||d¬« y)a¡
|
|||
|
|
Runs prediction trials for a trained model with varying thresholds to maximize
|
|||
|
|
the specified metric. The search space for the threshold is traversed linearly
|
|||
|
|
from 0 to 1 in `n_trials` steps. Results are displayed in a table on `stdout`
|
|||
|
|
(the corresponding API call to `spacy.cli.find_threshold.find_threshold()`
|
|||
|
|
returns all results).
|
|||
|
|
|
|||
|
|
This is applicable only for components whose predictions are influenced by
|
|||
|
|
thresholds - e.g. `textcat_multilabel` and `spancat`, but not `textcat`. Note
|
|||
|
|
that the full path to the corresponding threshold attribute in the config has to
|
|||
|
|
be provided.
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/cli#find-threshold
|
|||
|
|
F) r r r! r" r# r r r ÚsilentN)r
|
|||
|
|
ÚloggerÚsetLevelÚloggingÚDEBUGr Úfind_threshold)
|
|||
|
|
r r r! r" r# r r$ r r r% s
|
|||
|
|
úYC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/cli/find_threshold.pyÚfind_threshold_clir. sI € ñ@ Ü � ‰ × Ñ œWŸ]™]Ô+Ü� ÔÜØØØØ#Ø ØØØ!Øö
|
|||
|
|
ó )r r r r' r' Úreturnc
|
|||
|
|
óŽ ‡‡— t ||¬« t j |« }|j « s"t j
|
|||
|
|
j
|
|||
|
|
d|d¬« t j | « } || j vr4t t j j || j ¬« « ‚| j |« }
|
|||
|
|
t |
|
|||
|
|
d« st t j « ‚t! |
|
|||
|
|
« t" k( r t j
|
|||
|
|
j% d« |s*t j
|
|||
|
|
j' d|› d |› d
|
|||
|
|
|› d �¬ « t) ||¬
|
|||
|
|
« } t+ | | « « } |j- d« }
|
|||
|
|
dt. t0 t2 f dt4 t0 dt6 dt. t0 t2 f fd„}dt. t0 t2 f dt4 t0 dt0 dt. t0 t2 f fˆfd„ Ši Šd|g|
|
|||
|
|
¢}d}t9 j: dd|« }t= t j> jA d|› g|¬« « |D �]_ }t j | | ‰| jB |djE |« « jG « ||« ¬« } t |
|
|||
|
|
d « r.tI | j |« d |tK |
|
|||
|
|
d « |
|
|||
|
|
|« « | jM | « }||vr@t j
|
|||
|
|
j
|
|||
|
|
d |› d �d t+ |jO « « › �d¬ « || ‰|< tQ ‰| t6 tR f« s%t j
|
|||
|
|
j
|
|||
|
|
d!|› d"�d¬« t= t j@ tU |d#« tU ‰| d#« g|¬« « �Œb tW ‰jO « ˆfd$„¬%« }tY t[ ‰j] « « « dk( r8t j
|
|||
|
|
j% d&tQ |
|
|||
|
|
t^ « r|d'v rd(nd)¬*« n%|s#t= d+tU |d,¬-« › d.|› d/‰| › d�« |‰| ‰fS )0a:
|
|||
|
|
Runs prediction trials for models with varying thresholds to maximize the specified metric.
|
|||
|
|
model (Union[str, Path]): Pipeline to evaluate. Can be a package or a path to a data directory.
|
|||
|
|
data_path (Path): Path to file with DocBin with docs to use for threshold search.
|
|||
|
|
pipe_name (str): Name of pipe to examine thresholds for.
|
|||
|
|
threshold_key (str): Key of threshold attribute in component's configuration.
|
|||
|
|
scores_key (str): Name of score to metric to optimize.
|
|||
|
|
n_trials (int): Number of trials to determine optimal thresholds.
|
|||
|
|
use_gpu (int): GPU ID or -1 for CPU.
|
|||
|
|
gold_preproc (bool): Whether to use gold preprocessing. Gold preprocessing helps the annotations align to the
|
|||
|
|
tokenization, and may result in sequences of more consistent length. However, it may reduce runtime accuracy due
|
|||
|
|
to train/test skew.
|
|||
|
|
silent (bool): Whether to print non-error-related output to stdout.
|
|||
|
|
RETURNS (Tuple[float, float, Dict[float, float]]): Best found threshold, the corresponding score, scores for all
|
|||
|
|
evaluated thresholds.
|
|||
|
|
)r' zEvaluation data not foundr )Úexits)ÚnameÚoptsÚscorerz—The `textcat` component doesn't use a threshold as it's not applicable to the concept of exclusive classes. All thresholds will yield the same results.zOptimizing for z for component 'z' with z trials.)Útitle)r ú.ÚconfigÚkeysÚvaluer0 c ób — |t j t j |dd | « |d < | S )a Set item in nested dictionary. Adapted from https://stackoverflow.com/a/54138200.
|
|||
|
|
config (Dict[str, Any]): Configuration dictionary.
|
|||
|
|
keys (List[Any]): Path to value to set.
|
|||
|
|
value (float): Value to set.
|
|||
|
|
RETURNS (Dict[str, Any]): Updated dictionary.
|
|||
|
|
Nr )Ú functoolsÚreduceÚoperatorÚgetitem)r8 r9 r: s r- Úset_nested_itemz'find_threshold.<locals>.set_nested_itemƒ s4 € ð KPŒ ×Ñœ×)Ñ)¨4°°¨9°fÓ=¸dÀ2¹hÑG؈
|
|||
|
|
r/ Úfull_keyc
|
|||
|
|
ó •— |d | vrNt j j d|› d|d g› d�d|d g› dt | j « « › �d¬« |d t |« dkD r ‰| |d |dd |« iS | |d iS )
|
|||
|
|
a, Filters provided config dictionary so that only the specified keys path remains.
|
|||
|
|
config (Dict[str, Any]): Configuration dictionary.
|
|||
|
|
keys (List[Any]): Path to value to set.
|
|||
|
|
full_key (str): Full user-specified key.
|
|||
|
|
RETURNS (Dict[str, Any]): Filtered dictionary.
|
|||
|
|
r zFailed to look up `z` in config: sub-key z not found.zMake sure you specified z: correctly. The following sub-keys are available instead: r ©r6 Útextr2 N)ÚwasabiÚmsgÚfailÚlistr9 Úlen)r8 r9 rA Ú
|
|||
|
|
filter_configs €r- rJ z%find_threshold.<locals>.filter_config� s» ø€ ð �‰7˜&Ñ Ü �J‰J�O‰OØ+¨H¨:Ð5JÈDÐQRÉGÈ9È+ÐU`ÐaØ/°°a±° ¨{Ð:tܘŸ ™ ›
|
|||
|
|
Ó&Ð'ð)àð
|
|||
|
|
ô
|
|||
|
|
ð
|
|||
|
|
�‰GÜ�4‹y˜1Š}ñ # 6¨$¨q©'¡?°D¸¸°H¸hÓGð
|
|||
|
|
ð
|
|||
|
|
𠘘Q™‘ ð
|
|||
|
|
ð
|
|||
|
|
r/ Ú
|
|||
|
|
components)é
|
|||
|
|
rL r Ú Threshold)Úwidths)r8 ÚcfgzFailed to look up score `z` in evaluation results.z`Make sure you specified the correct value for `scores_key`. The following scores are available: rC zReturned score for key 'zG' is not numeric. Threshold optimization only works for numeric scores.é c ó •— ‰| S )N© )ÚkeyÚscoress €r- ú<lambda>z find_threshold.<locals>.<lambda>Õ s ø€ ¸Àºr/ )rS z?All scores are identical. Verify that all settings are correct.)Ú cats_macro_fÚ cats_micro_fÚ z\Use `cats_macro_f` or `cats_micro_f` when optimizing the threshold for `textcat_multilabel`.)r6 rD z
|
|||
|
|
Best threshold: é )Úndigitsz with z
|
|||
|
|
value of )0r r
|
|||
|
|
Ú ensure_pathr rE rF rG Ú
|
|||
|
|
load_modelÚcomponent_namesÚAttributeErrorr ÚE001ÚformatÚget_pipeÚhasattrÚE1045Útyper
|
|||
|
|
ÚwarnÚinfor rH Úsplitr Ústrr r ÚfloatÚnumpyÚlinspaceÚprintÚtablesÚrowr8 ÚjoinÚcopyÚsetattrÚgetattrÚevaluater9 Ú
|
|||
|
|
isinstanceÚintÚroundÚmaxrI ÚsetÚvaluesr )r r r! r" r# r r r r' ÚnlpÚpipeÚcorpusÚ dev_datasetÚ config_keysr@ Úconfig_keys_fullÚtable_col_widthsÚ
|
|||
|
|
thresholdsÚ thresholdÚ eval_scoresÚbest_thresholdrJ rT s @@r- r, r, G s ù€ ô: ˆg˜fÕ%Ü× Ñ Ó+€IØ × Ñ Ô Ü�
|
|||
|
|
‰
|
|||
|
|
�‰Ð3°YÀaˆÔHÜ
|
|||
|
|
�/‰/˜%Ó
|
|||
|
|
€Cà˜×+Ñ+Ñ+Ü Ü �K‰K× Ñ I°C×4GÑ4GÐ Ó Hó
|
|||
|
|
ð
|
|||
|
|
ð �<‰<˜ Ó "€DÜ �4˜Ô "Ü œVŸ\™\Ó*Ð*ä ˆDƒz”_Ò$Ü�
|
|||
|
|
‰
|
|||
|
|
�‰ð
|
|||
|
|
Mô
|
|||
|
|
ñ
|
|||
|
|
�
|
|||
|
|
‰
|
|||
|
|
�‰Ø# J <Ð/?À ¸{È'ÐRZÐQ[ð \ð ð ô
|
|||
|
|
ô �I¨LÔ
|
|||
|
|
9€FÜ‘v˜c“{Ó#€KØ ×%Ñ% cÓ*€Kð
|
|||
|
|
Ü”Sœ#�X‘ð
|
|||
|
|
Ü&*¬3¡ið
|