Skip to main content

Memory Issues and Light Database for Java ME

5 replies [Last post]
albert_kam
Offline
Joined: 2007-11-26
Points: 0

Hi all !

I've been using Apache Derby for my CDC app using Creme JVM. So far it works fine with small data (around 8 tables with total of 1000 small records of data). I've been testing it with around 26MB of user memory available in my pocket pc device.

Later on i tried csv processing for synchronization, downloading from wifi, extraction, etc, and it fails with OutOfMemoryError :-) I dont know if the available user memory is too small (I'm using ipaq messenger 6965, with 48MB user memory, the WM5 left me with 28MB to use) or is it my app or my database that consumes too much memory. I've been looking info about ipaq 112, which they say could have around 192MB of user memory, hee hee ..

I wonder too if there are other database with smaller memory print for mobile application ? I've been interested with db4o, but still unsure how to do relational things with it, i'm still new to it. I'm going to need simple joins.

If only Apache Derby got a simpler version for CDC app, which supports only basic sql operations and with smaller memory print. :-)

What do you think of this ? Please share your thoughts !

Regards,
Albert Kam

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
terrencebarr
Offline
Joined: 2004-03-04
Points: 0

Albert,

Interesting results. Please keep us posted, I'm sure there will be others who will benefit from your findings.

Regards,

-- Terrence

albert_kam
Offline
Joined: 2007-11-26
Points: 0

Hi,

I've been into some stuffs, and i made a post to the apache derby user mailing list at http://www.nabble.com/Derby-Memory-Issues-in-CDC-to17996719.html

I'll post for more updates

Regards,
Albert Kam

brucejh
Offline
Joined: 2009-01-26
Points: 0

Hi Albert,

I know this is several months after your questions, but I thought I would post this anyway because there are so many people who have asked the same question, which we use to ask. Now we have a better option to share.

We have been working in the mobile CDC and CLDC arena since 2000. Most of the options for a db have not been that great. We have used Derby and JavaDB and really have tested all of the SQL and Object DB's for CLDC and CDC enviornments. We were using Pointbase, which supported both environments, but then IBM bought the parent company and shelved the product. But even Pointbase was not well tested and had errors on SQL syntax.

We finally go tired of these types of issues and started developing a new db. We are going to launch this db for ourselves and make it available to others at a small fee in Feb 2009. This new db has been tested more thoroughly than any of the other options we have seen in the CLDC market and at this time we don’t believe there is any other good SQL option. We have used over 2000 test points on each platform that include multiple versions of Windows Mobile, Blackberry, Symbian, Android, Brew, LiMo, Mobilinux and with many JVM's including Aplix - JBlend, Esmertec - Jbed, Mysaifu, KAFFE, NSIcomp (CreMe), and IBM J9.

In Feb 2009, we will have tested versions for CLDC (for both 1.0 and 1.1 as well as for the RMS file system with an extension for unlimited data storage, JSR-75 file system and Blackberry’s native PerstObject file systems), CDC, Android and desktops.

The DB is only about 107 K (all of the other options are 3M+), it support a large portion of the SQL92 standards and provides triggers, views, most types of joins, embedded JDBC driver, auto-commit, encryption, uencoding (UTF-8) with language-specific sort templates, ACID compliance, etc. In the future, we will add capabilities for XML syntax and object syntax processing as well as synchronization tools and integration with all of the major development tools.

The DB can run in physical storage mode or in-memory mode with the option to only load tables into memory as needed. On a mobile phone (CLDC), it can process select transactions with 200 records and where clause parameters in less than 1/2 second and can join 10,000 records in about 3 seconds and only using about 350K of memory. These performance benchmarks are 10x to 250x better than what we have seen from other DB's in this environment and we have tuned the DB performance based upon a variety of internal and academic research.

Thus, we feel that we finally have a reasonable option for managing persistent data on CLDC phones while being able to use the same DB for applications on other mobile platforms and desktops. We are going to be using this as the basis for all of our future business and consumer applications.

If you would like to know more about it, let me know. We are going to make it available to other companies via a new company called Zoduna Corp (web site is not up yet). bruce.howard@zoduna.com

sfitzjava
Offline
Joined: 2003-06-15
Points: 0

There is also iAnywhere by sybase. I've not used it, but it may work for your needs.

Also here is a blog Terrence did a while back that has some links/info about mobileDBs
http://weblogs.java.net/blog/terrencebarr/archive/2007/12/databases_for_...

You may want to make sure you are not trying to load the objects into a memory array that expands (such as Vector) and not freeing the references. If the CSV conversion is not freeing the objects it creates once its done with it you could be seeing a memoryleak.

Regards,
-Shawn

albert_kam
Offline
Joined: 2007-11-26
Points: 0

Hi,

Thanks for your reply. I wish i knew Terrence's blog earlier, very helpful and up to date.

Anyway, i try out the monitoring tool provided by Creme, and the Apache Derby's memory usage is quite impressive. I tweaked with pageSize and pageCacheSize using derby.properties, and later on I tried inserting around eight hundreds of records with no significant increase of memory. Good news !

It seems that we're still going for the embedded RDBMS, rather than switching to ODBMS.

I'll be testing with a larger set of data in the poor pocket pc. Wish me luck ..

Regards,
Albert Kam