64 lines
11 KiB
Plaintext
64 lines
11 KiB
Plaintext
|
Ë
|
|||
|
|
=û g«% ã ó¸ — U d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z m
|
|||
|
|
Z
|
|||
|
|
d dlZd dlmZ ddlmZmZ ddlmZ dd lmZmZmZmZ d
|
|||
|
|
d lm Z m Z m Z m Z eeeeeed œZ e e!e d
|
|||
|
|
e
|
|||
|
|
e f f e"d< dZ#dZ$ G d„ de!e« Z% e jL d« e d
|
|||
|
|
dd¬« e dddd¬« e ddd d ¬ « e d
|
|||
|
|
d d d!¬ « e d"d#d$d%¬ « e dd&d'd(d)¬ « e d"d*d+d,¬ « e d"d-d.d/¬ « e e#d0d1d2 e'e jQ « « › �¬ « e dd3d4d5d¬« e dd6d7d8¬ « e dd9d:d;¬ « f d<e!d=ed>e%d?e)d@e*dAe e! dBe*dCe*dDe!dEe e dFe e! dGe*fdH„« Z+dId
|
|||
|
|
d"dd"d"ddd"dddJœ d<ed=e
|
|||
|
|
e!ef d>e!d?e)d@e*dAe e! dBe*dCe*dDe!dEe e dFe e! dGe*dKe*dLe e dMdf dN„Z,dOedPe!dMdfdQ„Z-dOedRedPe!dMdfdS„Z.dTe!dMe e! fdU„Z/dLed<ed=e
|
|||
|
|
e!ef d>e!dDe!dEe e f dV„Z0d<efdW„Z1y)Xé N)ÚEnum)ÚPath)ÚAnyÚCallableÚIterableÚMappingÚOptionalÚUnion)ÚPrinteré )ÚDocÚDocBin)Ú docs_to_json)Úconll_ner_to_docsÚconllu_to_docsÚ iob_to_docsÚ json_to_docsé )ÚArgÚOptÚappÚwalk_directory)Ú conllubioÚconlluÚconllÚnerÚiobÚjson.Ú
|
|||
|
|
CONVERTERSÚauto)r c ó — e Zd ZdZdZy)Ú FileTypesr ÚspacyN)Ú__name__Ú
|
|||
|
|
__module__Ú __qualname__r r# © ó úRC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/cli/convert.pyr" r" * s „ Ø €DØ �Er( r" ÚconvertzInput file or directoryT)ÚhelpÚexistsú-z!Output directory. '-' for stdout.)r+ Ú
|
|||
|
|
allow_dashr, r# z --file-typez-tzType of data to produce)r+ z --n-sentsz-nz*Number of sentences per doc (0 to disable)Fz --seg-sentsz-sz Segment sentences (for -c ner)z--modelz--basez-bzQTrained spaCy pipeline for sentence segmentation to use as base (for --seg-sents)z --morphologyz-mz#Enable appending morphology to tagsz--merge-subtokensz-TzMerge CoNLL-U subtokensz --converterz-cz Converter: z --ner-mapz-nmz6NER tag mapping (as JSON-encoded dict of entity types)z--langz-lz Language (if tokenizer required)z
|
|||
|
|
--concatenatez-Cz#Concatenate output to a single fileÚ
|
|||
|
|
input_pathÚ
|
|||
|
|
output_dirÚ file_typeÚn_sentsÚ seg_sentsÚmodelÚ
|
|||
|
|
morphologyÚmerge_subtokensÚ converterÚner_mapÚlangÚ concatenatec óô — t | « } |t d« k( rdn|}|dk( } t | ¬« }
|
|||
|
|
t |
|
|||
|
|
|| « }t |
|
|||
|
|
| ||j || « t | ||j ||||||| |
|
|||
|
|
| | |
|
|||
|
|
¬« y)aµ
|
|||
|
|
Convert files into json or DocBin format for training. The resulting .spacy
|
|||
|
|
file can be used with the train command and other experiment management
|
|||
|
|
functions.
|
|||
|
|
|
|||
|
|
If no output_dir is specified and the output format is JSON, the data
|
|||
|
|
is written to stdout, so you can pipe them forward to a JSON file:
|
|||
|
|
$ spacy convert some_file.conllu --file-type json > some_file.json
|
|||
|
|
|
|||
|
|
DOCS: https://spacy.io/api/cli#convert
|
|||
|
|
r- ©Úno_print) r1 r2 r3 r4 r5 r6 r7 r8 r9 r: ÚsilentÚmsgN)r r Ú_get_converterÚverify_cli_argsÚvaluer* )r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r> r? s r) Ú convert_clirC / sŽ € ô8 �jÓ!€JØ*4¼¸S» Ò*A¡3Àz€JØ
|
|||
|
|
˜3Ñ
|
|||
|
|
€FÜ
|
|||
|
|
˜6Ô
|
|||
|
|
"€CÜ ˜s I¨zÓ:€IÜ�C˜ Z°· ± À)ÈWÔUÜ ØØØ—/‘/ØØØØ Ø'ØØØ
|
|||
|
|
Ø ØØ ö r( r ) r1 r2 r3 r4 r5 r6 r8 r9 r: r> r? r> r? Úreturnc ó® — t | « } |
|
|||
|
|
s t | ¬« }
|
|||
|
|
| �t j | « nd } g }t | |« D ][ }|j dd¬« 5 }|j
|
|||
|
|
« }d d d « t | } ||||||
|
|||
|
|
|| | ¬« }|j ||f« Œ] | r7t j j |D ��cg c] \ }}|‘Œ c}}« }| |fg}|D ]æ \ }}|dk( rt |« g}t |« }n(t |d¬« }t |« }|j « }|d k( r
|
|||
|
|
t! ||« Œ]|| k7 r2|j# | « }t |« |j% d
|
|||
|
|
|› �« z }n/t |« |j& d z }|j% d
|
|||
|
|
|› �« }t) |||« |
|
|||
|
|
j+ d |› d
|
|||
|
|
|› �« Œè y # 1 sw Y �Œ^xY wc c}}w )Nr< Úrzutf-8©Úencoding)r2 r3 Úappend_morphologyr6 r9 r4 r= r8 r T)ÚdocsÚstore_user_datar- ú.éÿÿÿÿzGenerated output file (z
|
|||
|
|
documents): )r r ÚsrslyÚ read_jsonr ÚopenÚreadr ÚappendÚ itertoolsÚchainÚ
|
|||
|
|
from_iterabler Úlenr Úto_bytesÚ_print_docs_to_stdoutÚ relative_toÚ with_suffixÚpartsÚ_write_docs_to_fileÚgood)r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r> r? Ú doc_filesÚ input_locÚinfileÚ
|
|||
|
|
input_dataÚfuncrJ Ú_Úall_docsÚdataÚlen_docsÚdbÚsubpathÚ output_files r) r* r* c sß € ô" �jÓ!€J٠ܘvÔ&ˆØ*1Ð*=Œe�o‰o˜gÔ&À4€GØ€IÜ# J° Ö:ˆ Ø
|
|||
|
|
�^‰^˜C¨'ˆ^Ô
|
|||
|
|
2°fØ Ÿ™› ˆJ÷ 3ô ˜)Ñ$ˆÙØ ØØ Ø (Ø +ØØØØô
|
|||
|
|
|
|||
|
|
ˆð ×ј) TÐ*Õ+ð! ;ñ" Ü —?‘?×0Ñ0ÁiÔ1PÁi¹7¸1¸d²$ÀiÒ1PÓQˆØ (Ð+Ð,ˆ Û$‰ˆ �4Ø ˜Ò Ü Ó&Ð'ˆDܘ4“y‰Hä˜T°4Ô8ˆBܘ2“wˆHØ—;‘;“=ˆDØ ˜Ò Ü ! $¨ Õ 2à˜JÒ&Ø#×/Ñ/°
|
|||
|
|
Ó;�Ü " :Ó .°×1DÑ1DÀqÈÈ À_Ó1UÑ U‘ ä " :Ó .°· ± ÀÑ1DÑ D� Ø )× 5Ñ 5¸¸)¸°oÓ F� Ü k°9Ô =Ø �H‰HÐ.¨x¨j¸
|
|||
|
|
ÀkÀ]ÐSÕ Tñ% %÷' 3Ñ
|
|||
|
|
2üó" 2Qs ÁGÂ? G
|
|||
|
|
ÇG re Ú output_typec óŽ — |dk( rt j d| « y t j j j | « y )Nr r- )rN Ú
|
|||
|
|
write_jsonÚsysÚstdoutÚbufferÚwrite)re rj s r) rX rX ¢ s2 € Ø�fÒ Ü
|
|||
|
|
×ј˜dÕ#ä �
|
|||
|
|
‰
|