Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

export topicmap without data #339

Closed
GoogleCodeExporter opened this issue Mar 16, 2015 · 14 comments
Closed

export topicmap without data #339

GoogleCodeExporter opened this issue Mar 16, 2015 · 14 comments
Labels
Component-Ontopoly Enhancement GoogleCode Priority-Low Release5.2.0

Comments

@GoogleCodeExporter
Copy link 

We are writing a lot of tests that communicate with the topicmap. What would be 
great is to not have that connection to the database server for offline running 
of the tests. But its a lot of work to create all the topic types and data for 
each test that is already in the production topicmap.

Using a normal topicmap xml dump takes too long to load, 11sec on my core i7 
laptop. 

It would be great if we could only export the schema, TopicType, 
AssociationTypes etc. and no values/instances.

Original issue reported on code.google.com by idar.borlaug@gmail.com on 4 Jan 2011 at 3:08
@GoogleCodeExporter
Copy link
Author 

Seems the InMemoryTopicMapimplementation is not Transactional, would it be much 
work to make it Transactional?

This would make it a lot easier to use it for unit tests.

Original comment by idar.borlaug@gmail.com on 4 Jan 2011 at 3:45

@GoogleCodeExporter
Copy link
Author 

Making the InMemory implementation transactional would indeed be quite tricky, 
and would also hurt performance quite substantially. So I don't think this is 
feasible.

For unit tests, perhaps using the H2 embedded database would be easier?

Being able to export just the schema would indeed be a useful feature.

Original comment by lar...@gmail.com on 5 Jan 2011 at 6:38

  • Changed state: Accepted
  • Added labels: Component-Ontopoly

@GoogleCodeExporter
Copy link
Author 

Is there an easy way to import the XML into an H2 database or even derby? If i 
use Derby i only need to update the database when there are new topic types. If 
importing to H2 is quick enough then that might be good choice as well. Reading 
the xml today into inMemory implementation is slow, the XML file is 90MB.

Original comment by idar.borlaug@gmail.com on 5 Jan 2011 at 7:32

@GoogleCodeExporter
Copy link
Author 

I doubt importing into Derby/H2 is going to be any faster than the in-memory 
implementation. Anyway, you do it using the normal RDBMS backend code, but 
connect to H2/Derby instead of Oracle.

If you need the schema-only export function we could implement it for you, and 
charge the project for the hours. Shouldn't take long to do.

Original comment by lar...@gmail.com on 5 Jan 2011 at 7:37

@GoogleCodeExporter
Copy link
Author 

Lets wait a bit with implementing the schema-only export function. It might not 
be needed.

I do have some problems when i try to import the XTM file:
It seems the tables are not created, so it fails:
My test code:
TopicMapStoreIF store = new RDBMSTopicMapStore("db.H2OKS.props");

            // Get the new topic map object
        TopicMapIF tm = store.getTopicMap();

        TopicMapImporterIF reader = new XTMTopicMapReader(new File(SkrivbartTopicMapTestCase.class.getClassLoader().getResource( "tull.xtm" ).getFile()));
            reader.importInto(tm);
            System.err.println("Imported (id " + tm.getObjectId() + ").");

        store.commit();
        store.close();


Gives:
org.h2.jdbc.JdbcSQLException: Table "TM_ADMIN_SEQUENCE" not found; SQL 
statement:
select seq_count from TM_ADMIN_SEQUENCE where seq_name = ? for update 
[42102-148]

How do i triggerit to create the tables if they don't exist?

Original comment by idar.borlaug@gmail.com on 5 Jan 2011 at 8:05

@GoogleCodeExporter
Copy link
Author 

You need to create the tables by running the SQL create script, same as for 
Oracle. See $ONTOPIA/rdbms/setup

Original comment by lar...@gmail.com on 5 Jan 2011 at 8:11

@GoogleCodeExporter
Copy link
Author 

starting to get this to work, but now i have trouble importing the xtm file:
java.lang.NoSuchMethodError: 
com.thaiopensource.relaxng.SchemaFactory.setXMLReaderCreator(Lcom/thaiopensource
/relaxng/XMLReaderCreator;)V

Seems i have the wrong jing version?

dependencies loaded:
[INFO] ------------------------------------------------------------------------
[INFO] [dependency:tree {execution: default-cli}]
[INFO] no.kommune.bergen.portal:domain:jar:11.0-SNAPSHOT
[INFO] +- org.springframework:spring-context:jar:2.5.6:provided
[INFO] |  +- aopalliance:aopalliance:jar:1.0:provided
[INFO] |  \- commons-logging:commons-logging:jar:1.0.3:provided (version 
managed from 1.1.1)
[INFO] +- org.springframework:spring-beans:jar:2.5.6:provided
[INFO] +- org.springframework:spring-core:jar:2.5.6:provided
[INFO] +- com.escenic:escenic-io:jar:4.3-7:provided
[INFO] +- com.escenic:escenic-common:jar:4.3-7:provided
[INFO] +- no.kommune.bergen.portal:util:jar:11.0-SNAPSHOT:provided
[INFO] +- net.ontopia:ontopia-engine:jar:5.2.1:provided
[INFO] |  +- commons-dbcp:commons-dbcp:jar:1.2.2:provided
[INFO] |  |  \- commons-pool:commons-pool:jar:1.4:provided (version managed 
from 1.3)
[INFO] |  +- urbanophile:java-getopt:jar:1.0.9:provided
[INFO] |  +- org.slf4j:slf4j-api:jar:1.5.8:provided
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.5.8:provided
[INFO] |  +- log4j:log4j:jar:1.2.14:provided
[INFO] |  +- org.slf4j:jcl-over-slf4j:jar:1.5.8:provided
[INFO] |  +- antlr:antlr:jar:2.7.7:provided
[INFO] |  +- commons-collections:commons-collections:jar:3.2.1:provided
[INFO] |  +- apache-oro:jakarta-oro:jar:2.0.8:provided
[INFO] |  +- thaiopensource:jing:jar:20030619:provided
[INFO] |  +- net.sourceforge.jena:jena:jar:2.3:provided
[INFO] |  |  +- xerces:xercesImpl:jar:2.4.0:provided (version managed from 
2.7.1)
[INFO] |  |  +- com.ibm.icu:icu4j:jar:3.4.4:provided
[INFO] |  |  +- concurrent:concurrent:jar:1.3.4:provided (version managed from 
1.3.2)
[INFO] |  |  +- oro:oro:jar:2.0.8:provided
[INFO] |  |  +- stax:stax-api:jar:1.0:provided
[INFO] |  |  +- stax:stax:jar:1.1.1-dev:provided
[INFO] |  |  \- xml-apis:xml-apis:jar:1.3.02:provided
[INFO] |  +- trove:trove:jar:1.1-beta-5:provided (version managed from 2.1.1)
[INFO] |  +- backport-util-concurrent:backport-util-concurrent:jar:3.1:provided
[INFO] |  +- org.tmapi:tmapi:jar:1.0-SP1:provided
[INFO] |  \- lucene:lucene:jar:1.4.3:provided (version managed from 1.2)
[INFO] +- oracle:oracle-jdbc:jar:10.1.0.2.0:test
[INFO] +- commons-lang:commons-lang:jar:2.4:compile
[INFO] +- junit:junit:jar:4.8.1:test
[INFO] +- com.h2database:h2:jar:1.3.148:test
[INFO] +- idar:jing:jar:1.0:test
[INFO] \- com.aqris:easymock-wrapper:jar:0.8:test
[INFO]    \- easymock:easymock:jar:1.2_Java1.3:test

Original comment by idar.borlaug@gmail.com on 5 Jan 2011 at 9:05

@GoogleCodeExporter
Copy link
Author 

The idar:jing is the jing from ontopia.zip file

Original comment by idar.borlaug@gmail.com on 5 Jan 2011 at 9:11

@GoogleCodeExporter
Copy link
Author 

You've got two jings there: 

[INFO] |  +- thaiopensource:jing:jar:20030619:provided
[INFO] +- idar:jing:jar:1.0:test

That's probably one too many. (Yes, jing.jar is definitely the problem.)

Original comment by lar...@gmail.com on 5 Jan 2011 at 9:12

@GoogleCodeExporter
Copy link
Author 

Thanks, that fixed the problem :)

Now a new one, my XTM is broken it seems:
net.ontopia.utils.OntopiaRuntimeException: XML parsing problem: 
org.xml.sax.SAXParseException: unfinished element at: 
file:/home/idar/work/portal/domain/target/test-classes/tull.xtm:1062437:18
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.read(XTMTopicMapReader.java:232)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.readAll(XTMTopicMapReader.java:272)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.importInto(XTMTopicMapReader.java:291)
    at no.kommune.bergen.portal.SkrivbartTopicMapTestCase.createDatabase(SkrivbartTopicMapTestCase.java:61)
    at no.kommune.bergen.portal.SkrivbartTopicMapTestCase.setUp(SkrivbartTopicMapTestCase.java:32)
    at no.kommune.bergen.portal.domain.emne.VervTest.setUp(VervTest.java:10)
    at com.intellij.junit3.JUnit3IdeaTestRunner.doRun(JUnit3IdeaTestRunner.java:109)
    at com.intellij.junit3.JUnit3IdeaTestRunner.startRunnerWithArgs(JUnit3IdeaTestRunner.java:43)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
Caused by: org.xml.sax.SAXParseException: unfinished element
    at com.thaiopensource.relaxng.impl.PatternValidatorHandler.error(Unknown Source)
    at com.thaiopensource.relaxng.impl.PatternValidatorHandler.endElement(Unknown Source)
    at net.ontopia.topicmaps.xml.XTMValidatingContentHandler.endElement(XTMValidatingContentHandler.java:90)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.read(XTMTopicMapReader.java:226)
    ... 23 more
Caused by:
org.xml.sax.SAXParseException: unfinished element
    at com.thaiopensource.relaxng.impl.PatternValidatorHandler.error(Unknown Source)
    at com.thaiopensource.relaxng.impl.PatternValidatorHandler.endElement(Unknown Source)
    at net.ontopia.topicmaps.xml.XTMValidatingContentHandler.endElement(XTMValidatingContentHandler.java:90)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.read(XTMTopicMapReader.java:226)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.readAll(XTMTopicMapReader.java:272)
    at net.ontopia.topicmaps.xml.XTMTopicMapReader.importInto(XTMTopicMapReader.java:291)
    at no.kommune.bergen.portal.SkrivbartTopicMapTestCase.createDatabase(SkrivbartTopicMapTestCase.java:61)
    at no.kommune.bergen.portal.SkrivbartTopicMapTestCase.setUp(SkrivbartTopicMapTestCase.java:32)
    at no.kommune.bergen.portal.domain.emne.VervTest.setUp(VervTest.java:10)
    at com.intellij.junit3.JUnit3IdeaTestRunner.doRun(JUnit3IdeaTestRunner.java:109)
    at com.intellij.junit3.JUnit3IdeaTestRunner.startRunnerWithArgs(JUnit3IdeaTestRunner.java:43)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)

Original comment by idar.borlaug@gmail.com on 5 Jan 2011 at 9:22

Attachments:

@GoogleCodeExporter
Copy link
Author 

I would find it useful to be able to extract the ontology from an 
Ontopoly-created TM. I've considered writing  JSP for this but, as Lars points 
out, you need to know the Ontopoly ontology to get everything. The rewson that 
this functionality is imoortant to me is that I find developing an ontology to 
be an iterative process that benefits from using real data. At the end, I'd 
like to remove all of the instances and just leave the ontology to serve as a 
template.

Original comment by dan.sp...@gmail.com on 12 Jan 2011 at 2:36

@GoogleCodeExporter
Copy link
Author 

Have started on this one, but it appears that the fragment exporter may need to 
be modified a little for it to support this properly. Will continue working on 
it.

Original comment by lar...@gmail.com on 21 Feb 2011 at 2:13

  • Changed state: Started

@GoogleCodeExporter
Copy link
Author 

Implemented now.

Original comment by lar...@gmail.com on 18 Aug 2011 at 6:40

  • Changed state: Fixed

@GoogleCodeExporter
Copy link
Author

Original comment by qsieb...@gmail.com on 27 Jan 2012 at 10:47

  • Added labels: Release5.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component-Ontopoly Enhancement GoogleCode Priority-Low Release5.2.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@GoogleCodeExporter