Modeling and Managing Content Changes in Text Databases
- Modeling and Managing Content Changes in Text Databases
- Ipeirotis, Panagiotis G.
- Technical reports
- Computer Science
- Persistent URL:
- Columbia University Computer Science Technical Reports
- Part Number:
- Department of Computer Science, Columbia University
- Publisher Location:
- New York
- Large amounts of (often valuable) information are stored in web-accessible text databases. "Metasearchers" provide unified interfaces to query multiple such databases at once. For efficiency, metasearchers rely on succinct statistical summaries of the database contents to select the best databases for each query. So far, database selection research has largely assumed that databases are static, so the associated statistical summaries do not need to change over time. However, databases are rarely static and the statistical summaries that describe their contents need to be updated periodically to reflect content changes. In this paper, we first report the results of a study showing how the content summaries of 152 real web databases evolved over a period of 52 weeks. Then, we show how to use "survival analysis" techniques in general, and Cox's proportional hazards regression in particular, to model database changes over time and predict when we should update each content summary. Finally, we exploit our change model to devise update schedules that keep the summaries up to date by contacting databases only when needed, and then we evaluate the quality of our schedules experimentally over real web databases.
- Computer science
- Item views
text | xml
- Suggested Citation:
- Panagiotis G. Ipeirotis, Alexandros Ntoulas, Junghoo Cho, Luis Gravano, 2004, Modeling and Managing Content Changes in Text Databases, Columbia University Academic Commons, http://hdl.handle.net/10022/AC:P:29232.