chunkster that learns

chunkster that learns is a chunskter that:

1. chunks text automagically
2. records any changes humans make to output from 1.
3. updates, analyzes chunking statistics
4. learns to chunk better (improve step 1.)

when chunkster kicks ass, maybe we call it chunxa

of possible plugins to do now, improving chunkster will be key to good user experience.. good chunkster probably required before bothering to invite public testers..

note: "statistical" chunking is not statistical machine translation.. chunking is just chunking as per twext method.. apparently "statistical translation" is outperforming other machine translation methods.. so maybe there's a way we can use statistics to make chunking (and only chunking) easier..

• http://www.statmt.org/moses/
• http://en.wikipedia.org/wiki/Statistical_machine_translation
• http://en.wikipedia.org/wiki/Example-based_machine_translation The Example-based machine translation (EBMT) approach to machine translation is often characterized by its use of a bilingual corpus with parallel texts as its main knowledge base, at run-time. It is essentially a translation by analogy and can be viewed as an implementation of case-based reasoning approach of machine learning.