32 lines
19 KiB
Plaintext
32 lines
19 KiB
Plaintext
|
Ë
|
|||
|
|
?û gûG ã óš — d dl Z d dlZd dlmZ d dlmZ d dlmZ d„ Zd„ Z d„ Z
|
|||
|
|
e j j dg d ¢g d
|
|||
|
|
¢g d ¢d fg d
|
|||
|
|
¢g d¢g d¢dfg d¢g d¢g d¢dfg« d„ « Z
|
|||
|
|
e j j dddgg d¢g d¢g d¢dfdd gg d ¢g d ¢g d ¢d fg« d „ « Zd!„ Ze j j d"g d#¢« d$„ « Zy)%é N)ÚEnglish)Ú Sentencizer)ÚDocc óR — t | g d¢¬« }t d ¬« } ||« }|j d« sJ ‚|D �cg c] }|j ‘Œ }}|D �cg c] }|j ‘Œ }}|g d¢k( sJ ‚|g d¢k( sJ ‚t t
|
|||
|
|
|j « « dk( sJ ‚y c c}w c c}w )N)ÚHelloú!ÚThisÚisÚaÚtestú.©Úwords©Ú punct_charsÚ
|
|||
|
|
SENT_START©TFTFFFF)FTFFFFTé ©r r Úhas_annotationÚ
|
|||
|
|
is_sent_startÚ is_sent_endÚlenÚlistÚsents)Úen_vocabÚdocÚ sentencizerÚtÚ sent_startsÚ sent_endss úfC:\Users\garci\AppData\Roaming\Python\Python312\site-packages\spacy/tests/pipeline/test_sentencizer.pyÚtest_sentencizerr# s¨ € Ü
|
|||
|
|
|
|||
|
|
ˆhÒ LÔ
|
|||
|
|
M€CÜ ¨$Ô/€KÙ
|
|||
|
|
�cÓ
|
|||
|
|
€CØ × Ñ ˜lÔ +Ð+Ð +Ù,/Ó0©C q�1—?“?¨C€KÐ0Ù(+Ó,© 1�— “ ¨€IÐ,Ø ÒIÒ IÐIÐ IØ ÒGÒ GÐGÐ GÜ Œt�C—I‘I‹Ó 1Ò $Ð$Ñ $ùò 1ùÚ,s »B ÁB$c ó& — ddg} t « }|j d« |j | « D ]Z }|j d« sJ ‚|D �cg c] }|j ‘Œ }}|g d¢k( sJ ‚t t
|
|||
|
|
|j « « dk( rŒZJ ‚ |j | « D ]f }|j }|j d« sJ ‚|D �cg c] }|j ‘Œ }}|g d¢k( sJ ‚t t
|
|||
|
|
|j « « dk( rŒfJ ‚ y c c}w c c}w )NzHello! This is a test.zHi! This is a test.r r r r ) r Úadd_pipeÚpiper r r r r r )ÚtextsÚnlpr r r Úexs r" Útest_sentencizer_piper* s € Ø
|
|||
|
|
%Ð'<Ð =€EÜ
|
|||
|
|
‹)€C؇L�L� Ô Ø�x‰x˜ŽˆØ×!Ñ! ,Ô/Ð/Ð/Ù03Ó4±¨1�q—“°ˆ Ð4ØÒ MÒMÐMÐMÜ”4˜Ÿ ™ “?Ó# qÓ(Ð(Ð(ð ð
|
|||
|
|
�h‰h�uŽoˆØ�f‰fˆØ×!Ñ! ,Ô/Ð/Ð/Ù03Ó4±¨1�q—“°ˆ Ð4ØÒ MÒMÐMÐMÜ”4˜Ÿ ™ “?Ó# qÓ(Ð(Ð(ñ ùò 5ùò 5s Á D ÃDc ó> — dg} g d¢}g d¢}t « }|j d« | ||fD ]j }|j |« D ]T }|j d« sJ ‚|D �cg c] }|j ‘Œ }}t |« dk( r|g k( rŒCJ ‚t |« dkD rŒTJ ‚ Œl y c c}w )NÚ )r, r, r, )Úhir, z'This is a test. Here are two sentences.r, r r r )r r% r& r r r )Úone_empty_textÚmany_empty_textsÚsome_empty_textsr( r' r r r s r" Útest_sentencizer_empty_docsr1 & s« € Ø�T€NÚ#ÐÚPÐÜ
|
|||
|
|
‹)€C؇L�L� Ô Ø Ð"2Ð4DÓEˆØ—8‘8˜E–?ˆCØ×%Ñ% lÔ3Ð 3Ð3Ù47Ó8±C¨q˜1Ÿ?›?°CˆKÐ8Ü�3‹x˜1Š}Ø" bÓ(Ð(Ð(ä˜;Ó'¨!Ó+Ð+Ð+ñ
|
|||
|
|
#ñ Fùò 9s ÁBz#words,sent_starts,sent_ends,n_sents)r r r
|
|||
|
|
ÚTestr
|
|||
|
|
r
|
|||
|
|
Úok)TFFTFFT)FFTFFTTé )
|
|||
|
|
õ ¡ÚBuenu dÃar ÚHolaú,õ ¿u quéÚtalú?)
|
|||
|
|
TFFFTFFFFF)
|
|||
|
|
FFFTFFFFFTr )ú"ÚNicer r< ÚIÚamÚhappyr
|
|||
|
|
)TFFFTFFF)FFFTFFFTc óB — t | |¬« }t d ¬« } ||« }|j d« sJ ‚|D �cg c] }|j ‘Œ c}|k( sJ ‚|D �cg c] }|j ‘Œ c}|k( sJ ‚t t
|
|||
|
|
|j « « |k( sJ ‚y c c}w c c}w ©Nr r r r )r r r r! Ún_sentsr r r s r" Útest_sentencizer_complexrD 6 s� € ô: ˆh˜eÔ
|
|||
|
|
$€CÜ ¨$Ô/€KÙ
|
|||
|
|
�cÓ
|
|||
|
|
€CØ × Ñ ˜lÔ +Ð+Ð +Ù%(Ó )¡S ˆA�O‹O SÑ )¨[Ò 8Ð8Ð 8Ù#&Ó '¡3˜aˆA�M‹M 3Ñ '¨9Ò 4Ð4Ð 4Ü Œt�C—I‘I‹Ó 7Ò *Ð*Ñ *ùò *ùÚ 'ó ¹BÁB z/punct_chars,words,sent_starts,sent_ends,n_sentsú~r; )r ÚworldrF ÚAr
|