<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><blockquote type="cite">A solution to solving how to parse SQL would be not to parse it at all. <br> <br> We could consider the option of putting in a 'stored procedure' capable sql database as a requirement - and simply implementing<br> the vital parts of the broker as a collection of stored procedures or serverside code (maybe in java?). </blockquote><div><br class="webkit-block-placeholder"></div>Absolutely, and precisely: many smaller Collectors (implementations and instances) will *be* MySQL and Postgres powering them. Larger Collectors will likely end up doing some custom processing of the SQL, but there's no reason that they can't build off of the open source parsers to do that.</div><div><br><blockquote type="cite">Modern portals and document archives also offers us a challenge with searching inside structured documents - so my suggestion<br> is that we also offer support for searching in structures. <br> <br> We could start with describing a query language for Content Only (CO) based on SQL - and then build upon CO to add support <br> for Content and Structure (CAS)<br> <br> Andrew Trotman has described a basic query language NEXI supporting this here : <a class="moz-txt-link-freetext" href="http://www.cs.otago.ac.nz/postgrads/andrew/2004-4.pdf">http://www.cs.otago.ac.nz/postgrads/andrew/2004-4.pdf</a> <br> <br> I think the main contribution of the CAS part of our query language should be support of using XPATH from inside the query language.</blockquote><div><br class="webkit-block-placeholder"></div>Doing "document" and structured/XPATH like searching is out of scope for a Collector, and it's bordering on being out of scope for Atlas as a whole. I don't want to give the wrong impression, most of the value you get from this will still be possible, but Atlas is web search and entirely focused first on serving results to end user queries. Doing deep structural document data mining is definitely secondarily if relevant at all, and the Atlas architecture even makes it difficult, as we'll see in the next thread...</div><div><br class="webkit-block-placeholder"></div><div>Jer</div><div><br></div></body></html>