Latest @
Forum Controls
Spotlight Features
The Rich Engineering Heritage Behind Dependency Injection
Andrew McVeigh takes us on a tour of the rich heritage behind dependency injection, what it represents, and tells us why its here to stay.
Java, the OLPC, and community responsibility
The "One Laptop Per Child" project has a great device ready to ship, but there's no Java on there. Let's think about working together to put Java on OLPC!
[Teneo] Containment Move
At 1:31 PM on Jul 21, 2008, Maximilian Koegel wrote:
I use Teneo to make an EMF model persistent. The model has about 20.000
instances and 5-10 features per class. I use the following code to setup
Teneo:
final String hbStoreName = "modelStore";
HbDataStore dataStore =
HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
final Properties props = new Properties();
props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
props.setProperty(Environment.USER, "root");
props.setProperty(Environment.PASS, "pass");
props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
dataStore.setProperties(props);
// set epackages
dataStore.setEPackages(getUnicaseModelPackages());
dataStore.initialize();
// logger.debug("Using hibernate mapping: " +
// dataStore.getMappingXML());
String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
+ hbStoreName;
return URI.createURI(uriStr);
ResourceSet resourceSet = new ResourceSetImpl();
resource resource = resourceSet.createResource(URI.createURI(uriStr));
Lateron I call save() on the resource to save any changes I performed on
the model.
Whenever I initially add the model to the database (mysql) it takes
about 100 secs to do so. This seems quite slow to me.
Also when I change only one feature in one eObject it takes about the
same time.
Is there anything I can do to speed this up?
I already tried to disabled validation: I added an implementation for
EObjectValidator that will always return true for
validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
diagnostics, Map context).
Also I stumbled upon this:
http://www.elver.org/hibernate/hibernate_details.html#cutpaste
I am quite new to EMF and hibernate so I do not understand it
completely. To my understanding, it may be a performance hit to change
the container of an eObject, f.e. by adding it to another container.
This is something I do frequently for a certain containment feature.
What annotation should I add to the feature?
Is there anything else I could consider to improve the performance?
Thanks,
Maximilian
7 replies so far (
Post your own)
Re: [Teneo] Containment Move
Hi Maximilian,Which version of Teneo are you using? There was a specific issue which cost a lot of performance
with larger batches of objects before, this was solved two builds ago.
Changing one feature in one object should not cost that much performance...
Have you turned on hibernate logging so you can see if there is a lot of sql actions going on?
What puts load on the cpu: java or mysql?
Performance depends on many things (for example, are your memory settings for your test run large
neough?).
gr. Martin
Maximilian Koegel wrote:
> Hi,
> I use Teneo to make an EMF model persistent. The model has about 20.000
> instances and 5-10 features per class. I use the following code to setup
> Teneo:
> final String hbStoreName = "modelStore";
> HbDataStore dataStore =
> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>
> final Properties props = new Properties();
> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
> props.setProperty(Environment.USER, "root");
> props.setProperty(Environment.PASS, "pass");
> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>
> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
> dataStore.setProperties(props);
>
> // set epackages
> dataStore.setEPackages(getUnicaseModelPackages());
> dataStore.initialize();
>
> // logger.debug("Using hibernate mapping: " +
> // dataStore.getMappingXML());
> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
> + hbStoreName;
> return URI.createURI(uriStr);
> ResourceSet resourceSet = new ResourceSetImpl();
>
>
> resource resource = resourceSet.createResource(URI.createURI(uriStr));
> Lateron I call save() on the resource to save any changes I performed on
> the model.
> Whenever I initially add the model to the database (mysql) it takes
> about 100 secs to do so. This seems quite slow to me.
> Also when I change only one feature in one eObject it takes about the
> same time.
> Is there anything I can do to speed this up?
> I already tried to disabled validation: I added an implementation for
> EObjectValidator that will always return true for
> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
> diagnostics, Map context).
> Also I stumbled upon this:
> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
> I am quite new to EMF and hibernate so I do not understand it
> completely. To my understanding, it may be a performance hit to change
> the container of an eObject, f.e. by adding it to another container.
> This is something I do frequently for a certain containment feature.
> What annotation should I add to the feature?
> Is there anything else I could consider to improve the performance?
>
> Thanks,
> Maximilian
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal at springsite dot com - mtaal at elver dot org
Web: www.springsite.com - www.elver.org
Re: [Teneo] Containment Move
Hi Maximilian,Some extra remarks, moving an object to another container is not necessarily a performance hit. What
can cost performance though is that when using lists and moving an item from one (containment) list
to another then all the list indexes have to be recomputed (in the db). If the order in the lists
does not matter then you can also map your lists as an idbag. This can be done by setting the @IdBag
annotation on the list efeature.
Or use the teneo option: PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG to map all lists as a bag (with a
slighlty lesser performance than idbag but this only requires one option to be set).See
http://www.elver.org/hibernate/options.html
See also the hibernate chapter: 'Understanding Collection Performance'.
gr. Martin
Maximilian Koegel wrote:
> Hi,
> I use Teneo to make an EMF model persistent. The model has about 20.000
> instances and 5-10 features per class. I use the following code to setup
> Teneo:
> final String hbStoreName = "modelStore";
> HbDataStore dataStore =
> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>
> final Properties props = new Properties();
> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
> props.setProperty(Environment.USER, "root");
> props.setProperty(Environment.PASS, "pass");
> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>
> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
> dataStore.setProperties(props);
>
> // set epackages
> dataStore.setEPackages(getUnicaseModelPackages());
> dataStore.initialize();
>
> // logger.debug("Using hibernate mapping: " +
> // dataStore.getMappingXML());
> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
> + hbStoreName;
> return URI.createURI(uriStr);
> ResourceSet resourceSet = new ResourceSetImpl();
>
>
> resource resource = resourceSet.createResource(URI.createURI(uriStr));
> Lateron I call save() on the resource to save any changes I performed on
> the model.
> Whenever I initially add the model to the database (mysql) it takes
> about 100 secs to do so. This seems quite slow to me.
> Also when I change only one feature in one eObject it takes about the
> same time.
> Is there anything I can do to speed this up?
> I already tried to disabled validation: I added an implementation for
> EObjectValidator that will always return true for
> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
> diagnostics, Map context).
> Also I stumbled upon this:
> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
> I am quite new to EMF and hibernate so I do not understand it
> completely. To my understanding, it may be a performance hit to change
> the container of an eObject, f.e. by adding it to another container.
> This is something I do frequently for a certain containment feature.
> What annotation should I add to the feature?
> Is there anything else I could consider to improve the performance?
>
> Thanks,
> Maximilian
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal at springsite dot com - mtaal at elver dot org
Web: www.springsite.com - www.elver.org
Re: [Teneo] Containment Move
Hi Martin,> Some extra remarks, moving an object to another container is not
> necessarily a performance hit. What can cost performance though is that
> when using lists and moving an item from one (containment) list to
> another then all the list indexes have to be recomputed (in the db). If
> the order in the lists does not matter then you can also map your lists
> as an idbag. This can be done by setting the @IdBag annotation on the
> list efeature.
> Or use the teneo option: PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG to
> map all lists as a bag (with a slighlty lesser performance than idbag
> but this only requires one option to be set).See
> http://www.elver.org/hibernate/options.html
I tried setting the option PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG but
this results in an exception when generating the schema, see below.
Thanks,
Maximilian
!ENTRY org.eclipse.osgi 4 0 2008-07-22 00:39:36.034
!MESSAGE Application error
!STACK 1
java.lang.ClassCastException: org.eclipse.emf.ecore.impl.EReferenceImpl
at
org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.processOtMUni(OneToManyMapper.java:176)
at
org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.process(OneToManyMapper.java:58)
at
org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.processPAnnotatedEReference(FeatureMapper.java:166)
at
org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.process(FeatureMapper.java:99)
at
org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeature(EntityMapper.java:618)
at
org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeatures(EntityMapper.java:462)
at
org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processEntity(EntityMapper.java:306)
at
org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPAClass(HibernateMappingGenerator.java:215)
at
org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPersistentClasses(HibernateMappingGenerator.java:165)
at
org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generate(HibernateMappingGenerator.java:116)
at
org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generateToString(HibernateMappingGenerator.java:138)
at
org.eclipse.emf.teneo.hibernate.HbDataStore.mapEPackages(HbDataStore.java:612)
at
org.eclipse.emf.teneo.hibernate.HbSessionDataStore.mapModel(HbSessionDataStore.java:155)
at
org.eclipse.emf.teneo.hibernate.HbSessionDataStore.initialize(HbSessionDataStore.java:73)
at org.unicase.emfstore.storage.TeneoStorage.init(TeneoStorage.java:71)
at
org.unicase.emfstore.EmfStoreController.initServerSpace(EmfStoreController.java:109)
at
org.unicase.emfstore.EmfStoreController.start(EmfStoreController.java:75)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
>
> See also the hibernate chapter: 'Understanding Collection Performance'.
>
> gr. Martin
>
> Maximilian Koegel wrote:
>> Hi,
>> I use Teneo to make an EMF model persistent. The model has about
>> 20.000 instances and 5-10 features per class. I use the following code
>> to setup Teneo:
>> final String hbStoreName = "modelStore";
>> HbDataStore dataStore =
>> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>>
>> final Properties props = new Properties();
>> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
>> props.setProperty(Environment.USER, "root");
>> props.setProperty(Environment.PASS, "pass");
>> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
>> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>>
>> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
>> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
>> dataStore.setProperties(props);
>>
>> // set epackages
>> dataStore.setEPackages(getUnicaseModelPackages());
>> dataStore.initialize();
>>
>> // logger.debug("Using hibernate mapping: " +
>> // dataStore.getMappingXML());
>> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
>> + hbStoreName;
>> return URI.createURI(uriStr);
>> ResourceSet resourceSet = new ResourceSetImpl();
>>
>>
>> resource resource = resourceSet.createResource(URI.createURI(uriStr));
>> Lateron I call save() on the resource to save any changes I performed
>> on the model.
>> Whenever I initially add the model to the database (mysql) it takes
>> about 100 secs to do so. This seems quite slow to me.
>> Also when I change only one feature in one eObject it takes about the
>> same time.
>> Is there anything I can do to speed this up?
>> I already tried to disabled validation: I added an implementation for
>> EObjectValidator that will always return true for
>> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
>> diagnostics, Map context).
>> Also I stumbled upon this:
>> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
>> I am quite new to EMF and hibernate so I do not understand it
>> completely. To my understanding, it may be a performance hit to change
>> the container of an eObject, f.e. by adding it to another container.
>> This is something I do frequently for a certain containment feature.
>> What annotation should I add to the feature?
>> Is there anything else I could consider to improve the performance?
>>
>> Thanks,
>> Maximilian
>
>
Re: [Teneo] Containment Move
Hi Martin,thank you for your fast reply and for the great job you do with Teneo!
> Which version of Teneo are you using? There was a specific issue which
> cost a lot of performance with larger batches of objects before, this
> was solved two builds ago.
I was still on the release (1.0.0 (Wed, 11 Jun 2008 -- 19:28 (-0400)). I
just tested the current maintenance release and performance is really
improved by about 6x. To save 15.000 elements initially still takes 90
secs but updating only about 8 secs now.
> Have you turned on hibernate logging so you can see if there is a lot of
> sql actions going on?
How do I do that?
> What puts load on the cpu: java or mysql?
First it is only java for a few seconds then its is java and mysql at
about 30 and 50 percent respectively.
> Performance depends on many things (for example, are your memory
> settings for your test run large neough?).
The memory is probably sufficient, it is at 1500MB.
Cheers,
Maximilian
>
> gr. Martin
>
> Maximilian Koegel wrote:
>> Hi,
>> I use Teneo to make an EMF model persistent. The model has about
>> 20.000 instances and 5-10 features per class. I use the following code
>> to setup Teneo:
>> final String hbStoreName = "modelStore";
>> HbDataStore dataStore =
>> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>>
>> final Properties props = new Properties();
>> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
>> props.setProperty(Environment.USER, "root");
>> props.setProperty(Environment.PASS, "pass");
>> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
>> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>>
>> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
>> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
>> dataStore.setProperties(props);
>>
>> // set epackages
>> dataStore.setEPackages(getUnicaseModelPackages());
>> dataStore.initialize();
>>
>> // logger.debug("Using hibernate mapping: " +
>> // dataStore.getMappingXML());
>> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
>> + hbStoreName;
>> return URI.createURI(uriStr);
>> ResourceSet resourceSet = new ResourceSetImpl();
>>
>>
>> resource resource = resourceSet.createResource(URI.createURI(uriStr));
>> Lateron I call save() on the resource to save any changes I performed
>> on the model.
>> Whenever I initially add the model to the database (mysql) it takes
>> about 100 secs to do so. This seems quite slow to me.
>> Also when I change only one feature in one eObject it takes about the
>> same time.
>> Is there anything I can do to speed this up?
>> I already tried to disabled validation: I added an implementation for
>> EObjectValidator that will always return true for
>> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
>> diagnostics, Map context).
>> Also I stumbled upon this:
>> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
>> I am quite new to EMF and hibernate so I do not understand it
>> completely. To my understanding, it may be a performance hit to change
>> the container of an eObject, f.e. by adding it to another container.
>> This is something I do frequently for a certain containment feature.
>> What annotation should I add to the feature?
>> Is there anything else I could consider to improve the performance?
>>
>> Thanks,
>> Maximilian
>
>
Re: [Teneo] Containment Move
This shouldn't happen ofcourse. Would it be possible for you to zip up your project and email it tome? Then I can check what goes wrong here.
gr. Martin
Maximilian Koegel wrote:
> Hi Martin,
>
>> Some extra remarks, moving an object to another container is not
>> necessarily a performance hit. What can cost performance though is
>> that when using lists and moving an item from one (containment) list
>> to another then all the list indexes have to be recomputed (in the
>> db). If the order in the lists does not matter then you can also map
>> your lists as an idbag. This can be done by setting the @IdBag
>> annotation on the list efeature.
>> Or use the teneo option: PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG to
>> map all lists as a bag (with a slighlty lesser performance than idbag
>> but this only requires one option to be set).See
>> http://www.elver.org/hibernate/options.html
> I tried setting the option PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG but
> this results in an exception when generating the schema, see below.
>
> Thanks,
> Maximilian
>
> !ENTRY org.eclipse.osgi 4 0 2008-07-22 00:39:36.034
> !MESSAGE Application error
> !STACK 1
> java.lang.ClassCastException: org.eclipse.emf.ecore.impl.EReferenceImpl
> at
> org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.processOtMUni(OneToManyMapper.java:176)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.process(OneToManyMapper.java:58)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.processPAnnotatedEReference(FeatureMapper.java:166)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.process(FeatureMapper.java:99)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeature(EntityMapper.java:618)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeatures(EntityMapper.java:462)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processEntity(EntityMapper.java:306)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPAClass(HibernateMappingGenerator.java:215)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPersistentClasses(HibernateMappingGenerator.java:165)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generate(HibernateMappingGenerator.java:116)
>
> at
> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generateToString(HibernateMappingGenerator.java:138)
>
> at
> org.eclipse.emf.teneo.hibernate.HbDataStore.mapEPackages(HbDataStore.java:612)
>
> at
> org.eclipse.emf.teneo.hibernate.HbSessionDataStore.mapModel(HbSessionDataStore.java:155)
>
> at
> org.eclipse.emf.teneo.hibernate.HbSessionDataStore.initialize(HbSessionDataStore.java:73)
>
> at org.unicase.emfstore.storage.TeneoStorage.init(TeneoStorage.java:71)
> at
> org.unicase.emfstore.EmfStoreController.initServerSpace(EmfStoreController.java:109)
>
> at
> org.unicase.emfstore.EmfStoreController.start(EmfStoreController.java:75)
> at
> org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
>
> at
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
>
> at
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
>
> at
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
>
> at
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
> at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
> at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
> at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
>
>
>>
>> See also the hibernate chapter: 'Understanding Collection Performance'.
>>
>> gr. Martin
>>
>> Maximilian Koegel wrote:
>>> Hi,
>>> I use Teneo to make an EMF model persistent. The model has about
>>> 20.000 instances and 5-10 features per class. I use the following
>>> code to setup Teneo:
>>> final String hbStoreName = "modelStore";
>>> HbDataStore dataStore =
>>> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>>>
>>> final Properties props = new Properties();
>>> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
>>> props.setProperty(Environment.USER, "root");
>>> props.setProperty(Environment.PASS, "pass");
>>> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
>>> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>>>
>>> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
>>> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
>>> dataStore.setProperties(props);
>>>
>>> // set epackages
>>> dataStore.setEPackages(getUnicaseModelPackages());
>>> dataStore.initialize();
>>>
>>> // logger.debug("Using hibernate mapping: " +
>>> // dataStore.getMappingXML());
>>> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
>>> + hbStoreName;
>>> return URI.createURI(uriStr);
>>> ResourceSet resourceSet = new ResourceSetImpl();
>>>
>>>
>>> resource resource = resourceSet.createResource(URI.createURI(uriStr));
>>> Lateron I call save() on the resource to save any changes I performed
>>> on the model.
>>> Whenever I initially add the model to the database (mysql) it takes
>>> about 100 secs to do so. This seems quite slow to me.
>>> Also when I change only one feature in one eObject it takes about the
>>> same time.
>>> Is there anything I can do to speed this up?
>>> I already tried to disabled validation: I added an implementation for
>>> EObjectValidator that will always return true for
>>> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
>>> diagnostics, Map context).
>>> Also I stumbled upon this:
>>> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
>>> I am quite new to EMF and hibernate so I do not understand it
>>> completely. To my understanding, it may be a performance hit to
>>> change the container of an eObject, f.e. by adding it to another
>>> container. This is something I do frequently for a certain
>>> containment feature. What annotation should I add to the feature?
>>> Is there anything else I could consider to improve the performance?
>>>
>>> Thanks,
>>> Maximilian
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal at springsite dot com - mtaal at elver dot org
Web: www.springsite.com - www.elver.org
Re: [Teneo] Containment Move
Hi Maximilian,The performance is better but I think it should be faster than this.If you can zip up a test project
then I can try myself also.
Here is some tips on logging:
http://www.elver.org/hibernate/troubleshooting.html#runtime
gr. Martin
Maximilian Koegel wrote:
> Hi Martin,
> thank you for your fast reply and for the great job you do with Teneo!
>
>> Which version of Teneo are you using? There was a specific issue which
>> cost a lot of performance with larger batches of objects before, this
>> was solved two builds ago.
> I was still on the release (1.0.0 (Wed, 11 Jun 2008 -- 19:28 (-0400)). I
> just tested the current maintenance release and performance is really
> improved by about 6x. To save 15.000 elements initially still takes 90
> secs but updating only about 8 secs now.
>
>> Have you turned on hibernate logging so you can see if there is a lot
>> of sql actions going on?
> How do I do that?
>
>> What puts load on the cpu: java or mysql?
> First it is only java for a few seconds then its is java and mysql at
> about 30 and 50 percent respectively.
>
>> Performance depends on many things (for example, are your memory
>> settings for your test run large neough?).
> The memory is probably sufficient, it is at 1500MB.
>
> Cheers,
> Maximilian
>
>>
>> gr. Martin
>>
>> Maximilian Koegel wrote:
>>> Hi,
>>> I use Teneo to make an EMF model persistent. The model has about
>>> 20.000 instances and 5-10 features per class. I use the following
>>> code to setup Teneo:
>>> final String hbStoreName = "modelStore";
>>> HbDataStore dataStore =
>>> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>>>
>>> final Properties props = new Properties();
>>> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
>>> props.setProperty(Environment.USER, "root");
>>> props.setProperty(Environment.PASS, "pass");
>>> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" + "model");
>>> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>>>
>>> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
>>> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
>>> dataStore.setProperties(props);
>>>
>>> // set epackages
>>> dataStore.setEPackages(getUnicaseModelPackages());
>>> dataStore.initialize();
>>>
>>> // logger.debug("Using hibernate mapping: " +
>>> // dataStore.getMappingXML());
>>> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
>>> + hbStoreName;
>>> return URI.createURI(uriStr);
>>> ResourceSet resourceSet = new ResourceSetImpl();
>>>
>>>
>>> resource resource = resourceSet.createResource(URI.createURI(uriStr));
>>> Lateron I call save() on the resource to save any changes I performed
>>> on the model.
>>> Whenever I initially add the model to the database (mysql) it takes
>>> about 100 secs to do so. This seems quite slow to me.
>>> Also when I change only one feature in one eObject it takes about the
>>> same time.
>>> Is there anything I can do to speed this up?
>>> I already tried to disabled validation: I added an implementation for
>>> EObjectValidator that will always return true for
>>> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
>>> diagnostics, Map context).
>>> Also I stumbled upon this:
>>> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
>>> I am quite new to EMF and hibernate so I do not understand it
>>> completely. To my understanding, it may be a performance hit to
>>> change the container of an eObject, f.e. by adding it to another
>>> container. This is something I do frequently for a certain
>>> containment feature. What annotation should I add to the feature?
>>> Is there anything else I could consider to improve the performance?
>>>
>>> Thanks,
>>> Maximilian
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal at springsite dot com - mtaal at elver dot org
Web: www.springsite.com - www.elver.org
Re: [Teneo] Containment Move
yes, of course, thanks for helping us out! I just send you an email withthe zipped project and some additional information.
Cheers,
Maximilian
Martin Taal wrote:
> This shouldn't happen ofcourse. Would it be possible for you to zip up
> your project and email it to me? Then I can check what goes wrong here.
>
> gr. Martin
>
> Maximilian Koegel wrote:
>> Hi Martin,
>>
>>> Some extra remarks, moving an object to another container is not
>>> necessarily a performance hit. What can cost performance though is
>>> that when using lists and moving an item from one (containment) list
>>> to another then all the list indexes have to be recomputed (in the
>>> db). If the order in the lists does not matter then you can also map
>>> your lists as an idbag. This can be done by setting the @IdBag
>>> annotation on the list efeature.
>>> Or use the teneo option: PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG to
>>> map all lists as a bag (with a slighlty lesser performance than idbag
>>> but this only requires one option to be set).See
>>> http://www.elver.org/hibernate/options.html
>> I tried setting the option PersistenceOptions.ALWAYS_MAP_LIST_AS_BAG
>> but this results in an exception when generating the schema, see below.
>>
>> Thanks,
>> Maximilian
>>
>> !ENTRY org.eclipse.osgi 4 0 2008-07-22 00:39:36.034
>> !MESSAGE Application error
>> !STACK 1
>> java.lang.ClassCastException: org.eclipse.emf.ecore.impl.EReferenceImpl
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.processOtMUni(OneToManyMapper.java:176)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.OneToManyMapper.process(OneToManyMapper.java:58)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.processPAnnotatedEReference(FeatureMapper.java:166)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.FeatureMapper.process(FeatureMapper.java:99)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeature(EntityMapper.java:618)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processFeatures(EntityMapper.java:462)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.EntityMapper.processEntity(EntityMapper.java:306)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPAClass(HibernateMappingGenerator.java:215)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.processPersistentClasses(HibernateMappingGenerator.java:165)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generate(HibernateMappingGenerator.java:116)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.mapper.HibernateMappingGenerator.generateToString(HibernateMappingGenerator.java:138)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.HbDataStore.mapEPackages(HbDataStore.java:612)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.HbSessionDataStore.mapModel(HbSessionDataStore.java:155)
>>
>> at
>> org.eclipse.emf.teneo.hibernate.HbSessionDataStore.initialize(HbSessionDataStore.java:73)
>>
>> at
>> org.unicase.emfstore.storage.TeneoStorage.init(TeneoStorage.java:71)
>> at
>> org.unicase.emfstore.EmfStoreController.initServerSpace(EmfStoreController.java:109)
>>
>> at
>> org.unicase.emfstore.EmfStoreController.start(EmfStoreController.java:75)
>> at
>> org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
>>
>> at
>> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
>>
>> at
>> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
>>
>> at
>> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
>>
>> at
>> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:585)
>> at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
>> at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
>> at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
>> at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
>>
>>
>>>
>>> See also the hibernate chapter: 'Understanding Collection Performance'.
>>>
>>> gr. Martin
>>>
>>> Maximilian Koegel wrote:
>>>> Hi,
>>>> I use Teneo to make an EMF model persistent. The model has about
>>>> 20.000 instances and 5-10 features per class. I use the following
>>>> code to setup Teneo:
>>>> final String hbStoreName = "modelStore";
>>>> HbDataStore dataStore =
>>>> HbHelper.INSTANCE.createRegisterDataStore(hbStoreName);
>>>>
>>>> final Properties props = new Properties();
>>>> props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
>>>> props.setProperty(Environment.USER, "root");
>>>> props.setProperty(Environment.PASS, "pass");
>>>> props.setProperty(Environment.URL, "jdbc:mysql://localhost/" +
>>>> "model");
>>>> props.setProperty(Environment.DIALECT,org.hibernate.dialect.MySQLInnoDBDialect.class.getName());
>>>>
>>>> props.setProperty(Environment.MAX_FETCH_DEPTH, "0");
>>>> props.setProperty(PersistenceOptions.INHERITANCE_MAPPING, "JOINED");
>>>> dataStore.setProperties(props);
>>>>
>>>> // set epackages
>>>> dataStore.setEPackages(getUnicaseModelPackages());
>>>> dataStore.initialize();
>>>>
>>>> // logger.debug("Using hibernate mapping: " +
>>>> // dataStore.getMappingXML());
>>>> String uriStr = "hibernate://?" + HibernateResource.DS_NAME_PARAM + "="
>>>> + hbStoreName;
>>>> return URI.createURI(uriStr);
>>>> ResourceSet resourceSet = new ResourceSetImpl();
>>>>
>>>>
>>>> resource resource = resourceSet.createResource(URI.createURI(uriStr));
>>>> Lateron I call save() on the resource to save any changes I
>>>> performed on the model.
>>>> Whenever I initially add the model to the database (mysql) it takes
>>>> about 100 secs to do so. This seems quite slow to me.
>>>> Also when I change only one feature in one eObject it takes about
>>>> the same time.
>>>> Is there anything I can do to speed this up?
>>>> I already tried to disabled validation: I added an implementation
>>>> for EObjectValidator that will always return true for
>>>> validate_EveryReferenceIsContained(EObject eObject, DiagnosticChain
>>>> diagnostics, Map context).
>>>> Also I stumbled upon this:
>>>> http://www.elver.org/hibernate/hibernate_details.html#cutpaste
>>>> I am quite new to EMF and hibernate so I do not understand it
>>>> completely. To my understanding, it may be a performance hit to
>>>> change the container of an eObject, f.e. by adding it to another
>>>> container. This is something I do frequently for a certain
>>>> containment feature. What annotation should I add to the feature?
>>>> Is there anything else I could consider to improve the performance?
>>>>
>>>> Thanks,
>>>> Maximilian
>>>
>>>
>
>