The Document as an Ergodic Markov Chain
New York : ACM Press
InProceedings 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 496-497
Part of book or chapter of book
Display more detailsDisplay less details
SW OZ DCC KI
SW OZ NICI KI
Proceedings 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
SubjectCognitive artificial intelligence
In recent years, statistical language models are being proposed as alternative to the vector space model. Viewing documents as language samples introduces the issue of defining a joint probability distribution over the terms. The present paper models a document as the result of a Markov process. It argues that this process is ergodic, which is theoretically plausible, and easy to verify in practice. The theoretical result is that the joint distribution can be easily obtained. This can also be applied for search resolutions other than the document level. We verified this in an experiment on query expansion demonstrating both the validity and the practicability of the method. This holds a promise for general language models.
Upload full text
Use your RU credentials (u/z-number and password) to log in with SURFconext to upload a file for processing by the repository team.