Japanese model

[vochicong]

Hi, I am interested to using CoreNLP (and DeepDive) with Japanese.
Are you working on Japanese?
And how can I start building a Japanese model?
How did you build Chinese or English models?

Thanks!

[manning]

No, we are not currently working on Japanese. The first requirement to make models for any language is labeled data to train models from. Most of the components in CoreNLP use supervised learning. Traditionally, the public availability of Japanese language corpora hasn't been very good, but, now, e.g., the Japanese Universal Dependencies corpora could be used to train several components (segmenter, POS, depparse). However, I still don't know of any usable Japanese NER data. The other requirement is somebody willing to do the work. In general, our expansions to other languages have occurred because somebody was interested in having the language available for some reason.

[vochicong]

@manning Thank you for your reply. Actually, I am an NLP newbie and still don't fully understand your valuable answer ;)
But I think that Kuromoji, a Japanese morphological analyzer implemented in Java, looks promising to be embedded in CoreNLP.

I will search for Japanese NER data and if I find one I will share it with you.

BTW, I enjoy your Natural Language Processing with Deep Learning course very much! Thanks for it too!

[vochicong]

I found jigg. It's said having similar interface to CoreNLP, actually including CoreNLP and Kuromoji, a Japanese tokenizer. The authors are inspired by CoreNLP, once tried to make an Japanese extension to CoreNLP, but later decided to make jigg for more flexibility.

For Japanese NER, they use JUMAN/KNP.

[AngledLuffa]

FWIW (mostly for archival reasons at this point) there are now Japanese models for stanfordnlp

https://stanfordnlp.github.io/stanfordnlp/models.html#human-languages-supported-by-stanfordnlp

[vochicong]

Thank you @AngledLuffa for your update.

[devanghingu]

@AngledLuffa @vochicong i just checked standford package and stanza. but still not support for process relation extraction(kbp)

[AngledLuffa]

There is no relation extraction model in Stanza