Skip to main content

Memory management on "real-world" J2ME devices

21 replies [Last post]
Anonymous

Hi J2ME Community,

At Playscape Games we are developing a J2ME game 'Arcadia' that, as it
nears completion, is increasingly bumping against out target minimum
heap memory size of 800K.

I am considering strategies for memory optimisation & reclaimation, and
would appreciate any advice or opinions from the community on several
distinct memory related issues, with respect to modern real-world
handset J2ME implementations:

1. Memory Fragmentation:

Do/which current generation handset suffer memory fragmenetation (ie do
not use a compacting garbage collection algorithm)? One clear way to
reduce memory is to unload sprites that are not being rendered. This
will not work well with heap that fragment, however.

2. Unreclaimable memory

I have heard talk of handsets where Images (and perhaps other UI object
like Fonts, Canvases, etc) are stored in separate memory regions and are
not garbage collected, ever. What handsets, if any, have such
unreclaimable objects?

3. Reducing memory impact of Record Store access at Load/Save time

The Record Store API reads/writes data as byte[] blocks, rather than
streams. Our current load/save implementation therefore naively
read/writes the entire level data into a single byte[] array, then
parses/writes it respectively. We observe in a spike in memory
consumption at load/save time, well above the general memory
requirements during play. The spike forces us to keep approx. the last
100K of heap aside, unused, to just serve for a second while the game is
saved. We are seeking a way to reduce the memory overhead incurred when
saving level, by using a stream or small bufffer that is regularly
flushed to the Record Store.

Have other J2ME game developers had memory issues at Load/Save time?
Does anyone have an effective strategy for optimizing heap use when
working with Record Stores?

Regards
Ben Hutchison

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Stefan Haustein

Hi,

I think for pure Java objects, this is not an issue, but some devices
are really messed up concerning freeing system resources. For instance,
on a 6680, you will run into an OutOfMemoryException after creating a
certain amount of Canvas objects, although they are no longer referenced
anywhere (even when calling GC manually).

I would not recommend pooling StringBuffers or similar, though.

Best regards,
Stefan Haustein

Eric Giguere wrote:
> Elliotte Harold wrote:
>> Eric Giguere wrote:
>>> In my experience it's very important to recycle objects whenever
>>> possible. Sometimes this means creating a cache for storing various
>>> types of objects. It can really cut down on the garbage collection.
>>
>> In J2ME perhaps that is still true. I don't know. In J2SE and J2EE, that
>> hasn't been true for years (though it was at one time).
>
> This is a Java ME list and believe me, caching/pooling is still needed
> on many (most?) devices.
>
> Eric
>
> ===========================================================================
>
> To unsubscribe, send email to listserv@java.sun.com and include in the
> body
> of the message "signoff KVM-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Eric Giguere

Stefan Haustein wrote:
> I would not recommend pooling StringBuffers or similar, though.

We found we had to do just that to get the garbage collector not to run
as often. Your mileage may vary, I say write your app normally (no
caching of pure Java objects) and then run some tests to see if you can
get performance improvements with the caching.

Eric

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Robin Chaddock

One thing I would recommend, is handling char[]s directly rather than
working with Strings.
The object overhead of all those String wrappers is significant in almost
all applications.

Take a typical 176 wide screen, with an approriately sized font you will be
able to fit approx. 30 characters per line.
Given that characters are stored as chars, this gives ~60 bytes of raw data
per line.

As well written text splitting routines split the input character sequences
into multiple seperate output character sequences to allow support for token
injection, new line, break permitted here, soft-hyphen, zero width space,
and other useful formatting chars), the best case scenario will be that you
have a String object wrapper for every ~60 bytes of character data.

Given that the overhead of a String object is typically ~30bytes, this means
your text is being stored at ~66% efficiency.

There are 2 down sides to this approach,
1) Some encapsulation is lost, as obviously char[]s are mutable where-as
Strings are not.
2) Some API utility methods exist only for Strings (for example, there is no
StringBuffer.toCharArray()), leading to you on occasion having to
toString().toCharArray().

Obviously, there are other approaches. Storing text as bytes can be done.
However, it'll cause massive problems if you are ever required to localise
your application to a language that uses characters beyond the range of a
byte.

Another approach is for your text wrapping routine to leave the input
character sequence as-is, and simply return offsets for each line.
This however makes supporting (extremely useful) formatting characters and
token injection extremely difficult, if not impossible.

Ofcourse, the vast majority of J2ME applications are written with such
naivety that text wrapping *at all* can often be considered a luxury!

----- Original Message -----
From: "Eric Giguere"
To:
Sent: Thursday, May 17, 2007 11:51 AM
Subject: Re: Memory management on "real-world" J2ME devices

> Stefan Haustein wrote:
>> I would not recommend pooling StringBuffers or similar, though.
>
> We found we had to do just that to get the garbage collector not to run
> as often. Your mileage may vary, I say write your app normally (no
> caching of pure Java objects) and then run some tests to see if you can
> get performance improvements with the caching.
>
> Eric
>
> ===========================================================================
> To unsubscribe, send email to listserv@java.sun.com and include in the
> body
> of the message "signoff KVM-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".
>

________________________________________________________________________
E-mail is an informal method of communication and may be subject to data corruption, interception and unauthorised amendment for which I-play, a trading name of Digital Bridges Ltd will accept no liability. Therefore, it will normally be inappropriate to rely on information contained on e-mail without obtaining written confirmation.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

(C) 2005. I-play is a trademark and trading name of Digital Bridges Limited. All Rights Reserved.
________________________________________________________________________
This message has been checked for all known viruses by the
MessageLabs Virus Scanning Service. For further information visit
http://www.messagelabs.com/stats.asp

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Ben Hutchison

Hi Eric,

Can you give an example to illustrate your claim?

The garbage collector is a memory recycling system, so your claim that
the app should manage that recycling, instead of the GC, would be
strengthened if you could explain why/how, on J2ME in particular, the GC
does the recycling non-optimally.

As Eliotte mentioned, people kept advocating recycling mechanisms on
J2SE long after it was true, until a mass of benchmarks showed that GC
often outperformed app-managed recycling.

One conjecture is that the non-generational, non-concurrent GC in J2ME
has to lock and traverse the entire heap to collect even a single
object. If you have one type of objects that are being rapidly created &
destroyed, by recycling them you could conceivably clean up without a
full heap traversal.

Generational GC is an effective cure for this case. However, because it
partitions the heap into generations, it trades off memory efficiency
for performance, which may be undesirable for small memory sizes.

-Ben

Eric Giguere wrote:

> In my experience it's very important to recycle objects whenever
> possible. Sometimes this means creating a cache for storing various
> types of objects. It can really cut down on the garbage collection.
>
> Eric
>
> ===========================================================================
>
> To unsubscribe, send email to listserv@java.sun.com and include in the
> body
> of the message "signoff KVM-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".
>
>

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Eric Giguere

All I know is that caching works well for me on the BlackBerry. Without
caching you see the hourglass (which indicates garbage collection) a lot
more often.

Eric

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Daniel Rocha

Specific to Nokia's S60 devices, and possibily applicable to other Symbian devices as well: do not create multiple instances of Canvas, as it is a high-overhead object whose native implementation creates RWindow and RBackupWindow instances, which are heavyweight objects, in heap memory. Each new Canvas creates a new "RWindow" and this is know to exhaust memory pretty quickly.

I agree with previous poster that writing in big chunks is better than both saving everything at once or writing multiple small chunks. Perhaps you could also use a new Thread for I/O operations and save-as-you-go pattern. That would avoid the memory spike as well.

Images are problematic in some devices, I would only use PNG instead of JPEG, as they have smaller heap overhead.

I have a presentation about optimization for S60 devices, not exclusively for games, but it can be of some help. Contact me if you need it.

[]s
Daniel
Forum Nokia

________________________________

From: A mailing list for KVM discussion [mailto:KVM-INTEREST@JAVA.SUN.COM] On Behalf Of ext Joe Bowbeer
Sent: Monday, May 14, 2007 12:26 PM
To: KVM-INTEREST@JAVA.SUN.COM
Subject: Re: Memory management on "real-world" J2ME devices

Can you could share more details concerning what you've tried so far and where you've failed? Also, what talk have you heard concerning handsets with dedicated image storage? (Come on! Share and share alike!)

Here are two leads:

1. The MIDP DeviceDB, filled with data gathered by Jörg Pleumann's SysInfo MIDlet, contains some memory fragmentation results, but it appears to be off line now, and is possibly too far out of date for your needs:

http://sourceforge.net/projects/midpsysinfo/
http://kobjects.sourceforge.net/about.shtml
http://devicedb.kobjects.org/

2. I've managed to avoid some OutOfMemoryExceptions by implementing a streaming I/O interface to RMS. This saves some memory by avoiding excessive buffering in heap. I use 16K or 32K byte records (chunks).

(If only MIDlets had a randomly accessible scratchpad like iApplication's...)

--
Joe Bowbeer

On 5/12/07, Ben Hutchison < ben@playscapegames.com > wrote:

Hi J2ME Community,

At Playscape Games we are developing a J2ME game 'Arcadia' that, as it
nears completion, is increasingly bumping against out target minimum
heap memory size of 800K.

I am considering strategies for memory optimisation & reclaimation, and
would appreciate any advice or opinions from the community on several
distinct memory related issues, with respect to modern real-world
handset J2ME implementations:

1. Memory Fragmentation:

Do/which current generation handset suffer memory fragmenetation (ie do
not use a compacting garbage collection algorithm)? One clear way to
reduce memory is to unload sprites that are not being rendered. This
will not work well with heap that fragment, however.

2. Unreclaimable memory

I have heard talk of handsets where Images (and perhaps other UI object
like Fonts, Canvases, etc) are stored in separate memory regions and are
not garbage collected, ever. What handsets, if any, have such
unreclaimable objects?

3. Reducing memory impact of Record Store access at Load/Save time

The Record Store API reads/writes data as byte[] blocks, rather than
streams. Our current load/save implementation therefore naively
read/writes the entire level data into a single byte[] array, then
parses/writes it respectively. We observe in a spike in memory
consumption at load/save time, well above the general memory
requirements during play. The spike forces us to keep approx. the last
100K of heap aside, unused, to just serve for a second while the game is
saved. We are seeking a way to reduce the memory overhead incurred when
saving level, by using a stream or small bufffer that is regularly
flushed to the Record Store.

Have other J2ME game developers had memory issues at Load/Save time?
Does anyone have an effective strategy for optimizing heap use when
working with Record Stores?

Regards
Ben Hutchison

=========================================================================== To unsubscribe, send email to listserv@java.sun.com and include in the body of the message "signoff KVM-INTEREST". For general help, send email to listserv@java.sun.com and include in the body of the message "help".

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".
[att1.html]

Joe Bowbeer

Can you could share more details concerning what you've tried so far and
where you've failed? Also, what talk have you heard concerning handsets
with dedicated image storage? (Come on! Share and share alike!)

Here are two leads:

1. The MIDP DeviceDB, filled with data gathered by Jörg Pleumann's SysInfo
MIDlet, contains some memory fragmentation results, but it appears to be off
line now, and is possibly too far out of date for your needs:

http://sourceforge.net/projects/midpsysinfo/
http://kobjects.sourceforge.net/about.shtml
http://devicedb.kobjects.org/

2. I've managed to avoid some OutOfMemoryExceptions by implementing a
streaming I/O interface to RMS. This saves some memory by avoiding
excessive buffering in heap. I use 16K or 32K byte records (chunks).

(If only MIDlets had a randomly accessible scratchpad like
iApplication's...)

--
Joe Bowbeer

On 5/12/07, Ben Hutchison wrote:
>
> Hi J2ME Community,
>
> At Playscape Games we are developing a J2ME game 'Arcadia' that, as it
> nears completion, is increasingly bumping against out target minimum
> heap memory size of 800K.
>
> I am considering strategies for memory optimisation & reclaimation, and
> would appreciate any advice or opinions from the community on several
> distinct memory related issues, with respect to modern real-world
> handset J2ME implementations:
>
> 1. Memory Fragmentation:
>
> Do/which current generation handset suffer memory fragmenetation (ie do
> not use a compacting garbage collection algorithm)? One clear way to
> reduce memory is to unload sprites that are not being rendered. This
> will not work well with heap that fragment, however.
>
> 2. Unreclaimable memory
>
> I have heard talk of handsets where Images (and perhaps other UI object
> like Fonts, Canvases, etc) are stored in separate memory regions and are
> not garbage collected, ever. What handsets, if any, have such
> unreclaimable objects?
>
> 3. Reducing memory impact of Record Store access at Load/Save time
>
> The Record Store API reads/writes data as byte[] blocks, rather than
> streams. Our current load/save implementation therefore naively
> read/writes the entire level data into a single byte[] array, then
> parses/writes it respectively. We observe in a spike in memory
> consumption at load/save time, well above the general memory
> requirements during play. The spike forces us to keep approx. the last
> 100K of heap aside, unused, to just serve for a second while the game is
> saved. We are seeking a way to reduce the memory overhead incurred when
> saving level, by using a stream or small bufffer that is regularly
> flushed to the Record Store.
>
> Have other J2ME game developers had memory issues at Load/Save time?
> Does anyone have an effective strategy for optimizing heap use when
> working with Record Stores?
>
>
> Regards
> Ben Hutchison
>

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".
[att1.html]

Ben Hutchison

Joe Bowbeer wrote:

> Can you could share more details concerning what you've tried so far
> and where you've failed? Also, what talk have you heard concerning
> handsets with dedicated image storage? (Come on! Share and share alike!)

Sure. I have a summary of what I know/guess on my blog at:

http://benhutchison.wordpress.com/2007/04/23/j2me-memory-profiling/

-Ben

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Joe Bowbeer

Thanks.

One note: the object size does vary according to VM. Project Monty,
for example, only allocates 8 bytes per Object (initially).

It's hard to find published numbers for the various VM brands, or even
to determine the brand of VM employed by a particular handset. But
it's easy to determine the object sizes on a particular handset by
running a simple test. Then these figure can be combined with the
instance counts obtained from WTK to obtain more accurate memory usage
figures.

--
Joe Bowbeer

On 5/15/07, Ben Hutchison wrote:
> Joe Bowbeer wrote:
>
> > Can you could share more details concerning what you've tried so far
> > and where you've failed? Also, what talk have you heard concerning
> > handsets with dedicated image storage? (Come on! Share and share alike!)
>
> Sure. I have a summary of what I know/guess on my blog at:
>
> http://benhutchison.wordpress.com/2007/04/23/j2me-memory-profiling/
>
> -Ben
>

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Joe Bowbeer

PS - A couple of links for future reference.

This paper might be worth tracking down:

Vesa-Matti Hartikainen, Pasi P. Liimatainen, Tommi Mikkonen, "On
Mobile Java Memory Consumption," pdp, pp. 333-339, 14th Euromicro
International Conference on Parallel, Distributed, and Network-Based
Processing (PDP'06), 2006.

The Small Memory Software book was heading in the right direction. I
wish someone would extend this work specifically for Java ME:

http://www.smallmemory.com/

On 5/15/07, Joe Bowbeer wrote:
> Thanks.
>
> One note: the object size does vary according to VM. Project Monty,
> for example, only allocates 8 bytes per Object (initially).
>
> It's hard to find published numbers for the various VM brands, or even
> to determine the brand of VM employed by a particular handset. But
> it's easy to determine the object sizes on a particular handset by
> running a simple test. Then these figure can be combined with the
> instance counts obtained from WTK to obtain more accurate memory usage
> figures.
>
> --
> Joe Bowbeer
>
> On 5/15/07, Ben Hutchison wrote:
> > Joe Bowbeer wrote:
> >
> > > Can you could share more details concerning what you've tried so far
> > > and where you've failed? Also, what talk have you heard concerning
> > > handsets with dedicated image storage? (Come on! Share and share alike!)
> >
> > Sure. I have a summary of what I know/guess on my blog at:
> >
> > http://benhutchison.wordpress.com/2007/04/23/j2me-memory-profiling/
> >
> > -Ben
> >
>

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Eric Giguere

In my experience it's very important to recycle objects whenever
possible. Sometimes this means creating a cache for storing various
types of objects. It can really cut down on the garbage collection.

Eric

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Elliotte Harold

Eric Giguere wrote:
> In my experience it's very important to recycle objects whenever
> possible. Sometimes this means creating a cache for storing various
> types of objects. It can really cut down on the garbage collection.
>

In J2ME perhaps that is still true. I don't know. In J2SE and J2EE, that
hasn't been true for years (though it was at one time). Modern J2SE
garbage collectors are very good. Recycling objects there rarely helps
and may well hurt, unless there's a lot of class specific overhead
associated with each object such as a persistent network connection.
However for typical objects, pooling just isn't worth the trouble any more.

--
Elliotte Rusty Harold elharo@metalab.unc.edu
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Eric Giguere

Elliotte Harold wrote:
> Eric Giguere wrote:
>> In my experience it's very important to recycle objects whenever
>> possible. Sometimes this means creating a cache for storing various
>> types of objects. It can really cut down on the garbage collection.
>
> In J2ME perhaps that is still true. I don't know. In J2SE and J2EE, that
> hasn't been true for years (though it was at one time).

This is a Java ME list and believe me, caching/pooling is still needed
on many (most?) devices.

Eric

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Joe Bowbeer

On 5/16/07, Eric Giguere wrote:
>
> This is a Java ME list and believe me, caching/pooling is still needed
> on many (most?) devices.
>

Exactly.

The critical problem to avoid is running out of memory, which is
essentially unrecoverable (in all Java platforms), or getting too
close to the limit, which results in thrashing that renders the client
unusable.

Programming within these constraints is much more difficult (an order
of magnitude so?) than programming for the desktop or the server.
That's why they pay us the big bucks, right? :-)

--Joe

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Ben Hutchison

Joe Bowbeer wrote:

> The critical problem to avoid is running out of memory, which is
> essentially unrecoverable (in all Java platforms), or getting too
> close to the limit, which results in thrashing that renders the client
> unusable.

So app-managed recycling gives you certainty? Thats the benefit? Ie if I
have an allocated pool of a 1000 records, and 800 are in use, I know
exactly what Ive got left. Whereas, with the GC, by the time I get an
OutOfMemoryError, its kinda means "Sorry, you died!". And asking the
system for freeMemory() gets back some kind of semi-fictional number!

2 thoughts:

1. If you have parts of your app that cache non-essential data, you
could write a try/catch structure that traps OutOfMemoryErrors, and
dumps all caches in response. We do something very similar to this in
Arcadia.

2. The strategy in(1) isnt reliable for all cases, from my experience.
Either some handsets dont honour catch (OutOfMemoryError e) [and
NullPointer], or it happens in a non-user thread where you cannot trap
it anyway.

-Ben

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

Joe Bowbeer

On 5/16/07, Ben Hutchison wrote:
>
> Joe Bowbeer wrote:
>
> > The critical problem to avoid is running out of memory, which is
> > essentially unrecoverable (in all Java platforms), or getting too
> > close to the limit, which results in thrashing that renders the client
> > unusable.
>
> So app-managed recycling gives you certainty? Thats the benefit? Ie if I
> have an allocated pool of a 1000 records, and 800 are in use, I know
> exactly what Ive got left. Whereas, with the GC, by the time I get an
> OutOfMemoryError, its kinda means "Sorry, you died!". And asking the
> system for freeMemory() gets back some kind of semi-fictional number!
>
> 2 thoughts:
>
> 1. If you have parts of your app that cache non-essential data, you
> could write a try/catch structure that traps OutOfMemoryErrors, and
> dumps all caches in response. We do something very similar to this in
> Arcadia.
>
> 2. The strategy in(1) isnt reliable for all cases, from my experience.
> Either some handsets dont honour catch (OutOfMemoryError e) [and
> NullPointer], or it happens in a non-user thread where you cannot trap
> it anyway.
>
> -Ben
>
>

> So app-managed recycling gives you certainty? That's the benefit?

Right.

As for trapping OutOfMemory errors, I've done that as well and I agree that
it's a last-ditch effort. I suspect it's not significantly more effective
than wishful thinking.

Implementing a lot of features in a small amount of memory is a tough
problem. I don't believe anyone has found a universally applicable
solution. AFAIK, all the techniques for dealing with it need some tuning
and tweaking -- and sometimes kludging.

The result is that a lot of MIDlets cache objects and explicitly call
System.gc().

A lot of MIDlets will call gc() after freeing large chunks of memory or
before requiring large chunks of memory. Some MIDlets check freeMemory and
only call gc() when the result falls below a certain threshold. Some
MIDlets regulate the frequency of gc() calls by remembering the time of the
last gc() and only calling gc() again if a certain time interval has passed
(e.g., 1 second).

Note: System.gc() usually doesn't take very long (it's only a hint) but I
read reports a few years ago that one of the newer VMs was doing a full gc
each time gc() was called. So be careful.

I've found that some operations are likely to throw OutOfMemory too easily
-- before triggering a gc. HttpConnection and createImage are sometimes
like this (depends on the VM), so I've needed to free cached items and even
call gc() before opening HttpConnections and/or creating JPEG images in
order to avoid running out of memory.

--
Joe Bowbeer

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".
[att1.html]

Dhawal Ogale

Ben Hutchison said:
>>2. Unreclaimable memory
>>
>>I have heard talk of handsets where Images (and perhaps other UI object
>>like Fonts, Canvases, etc) are stored in separate memory regions and are
>>not garbage collected, ever. What handsets, if any, have such
>>unreclaimable objects?

I would be wary of loading any images on such a device :-)
However, fonts may not mean too much because MIDP supports so few fonts.
Also you don't want MIDP loading and unloading images behind the fonts as
this would entail a performance penalty.

Ben Hutchison said:
>>3. Reducing memory impact of Record Store access at Load/Save time
>>
>>The Record Store API reads/writes data as byte[] blocks, rather than
>>streams. Our current load/save implementation therefore naively
>>read/writes the entire level data into a single byte[] array, then
>>parses/writes it respectively. We observe in a spike in memory
>>consumption at load/save time, well above the general memory
>>requirements during play. The spike forces us to keep approx. the last
>>100K of heap aside, unused, to just serve for a second while the game is
>>saved. We are seeking a way to reduce the memory overhead incurred when
>>saving level, by using a stream or small bufffer that is regularly
>>flushed to the Record Store.

You could see a spike in memory because you possibly allocate a lot of small
objects. Consider allocating a few big objects instead of many small
objects. You could use techniques as using index inside a single big int
array instead of a thousand small 4 item int arrays.

That you are writing big records to RMS is not bad at all. There could be
nothing worse than writing large number of small records to RMS,
particularly if you are going to delete them and recreate new such records.
It's always better to not delete RMS records and not change the size of
records through updation, though updating with data exactly of the same size
works just fine.

-----Original Message-----
From: A mailing list for KVM discussion [mailto:KVM-INTEREST@JAVA.SUN.COM]
On Behalf Of Ben Hutchison
Sent: Saturday, May 12, 2007 9:49 PM
To: KVM-INTEREST@JAVA.SUN.COM
Subject: Memory management on "real-world" J2ME devices

Hi J2ME Community,

At Playscape Games we are developing a J2ME game 'Arcadia' that, as it
nears completion, is increasingly bumping against out target minimum
heap memory size of 800K.

I am considering strategies for memory optimisation & reclaimation, and
would appreciate any advice or opinions from the community on several
distinct memory related issues, with respect to modern real-world
handset J2ME implementations:

1. Memory Fragmentation:

Do/which current generation handset suffer memory fragmenetation (ie do
not use a compacting garbage collection algorithm)? One clear way to
reduce memory is to unload sprites that are not being rendered. This
will not work well with heap that fragment, however.

2. Unreclaimable memory

I have heard talk of handsets where Images (and perhaps other UI object
like Fonts, Canvases, etc) are stored in separate memory regions and are
not garbage collected, ever. What handsets, if any, have such
unreclaimable objects?

3. Reducing memory impact of Record Store access at Load/Save time

The Record Store API reads/writes data as byte[] blocks, rather than
streams. Our current load/save implementation therefore naively
read/writes the entire level data into a single byte[] array, then
parses/writes it respectively. We observe in a spike in memory
consumption at load/save time, well above the general memory
requirements during play. The spike forces us to keep approx. the last
100K of heap aside, unused, to just serve for a second while the game is
saved. We are seeking a way to reduce the memory overhead incurred when
saving level, by using a stream or small bufffer that is regularly
flushed to the Record Store.

Have other J2ME game developers had memory issues at Load/Save time?
Does anyone have an effective strategy for optimizing heap use when
working with Record Stores?

Regards
Ben Hutchison

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

captainfreedom
Offline
Joined: 2007-01-10
Points: 0

One thing to bear in mind is the garbage collector never runs unless there are some free cpu cycles available (i.e only when the currently running function finishes). For this reason it's not a good idea to create objects in your main game loop.

Ben Hutchison

meinterest@MOBILEANDEMBEDDED.ORG wrote:

>One thing to bear in mind is the garbage collector never runs unless there are some free cpu cycles available (i.e only when the currently running function finishes). For this reason it's not a good idea to create objects in your main game loop.
>[Message sent by forum member 'captainfreedom' (captainfreedom)]
>
I dont think this is correct. Its certaintly not the case in J2SE.

You can confirm this for yourself by running the following MIDlet, which
continously allocates objects in a tight loop. If the GC had to wait for
spare cycles (ie couldn't interrupt the loop), this Midlet should crash
very quickly, but in my testing it runs indefinitely, meaning the
created Vectors are being cleaned up.

public class GcTest extends MIDlet {
protected void startApp() throws MIDletStateChangeException {
while (true) {
new Vector(1000);
}
}

protected void destroyApp(boolean unconditional) throws
MIDletStateChangeException {
}

protected void pauseApp() {
}
}

I'd expect the main trigger for the garbage collector running is a low
memory condition. Ie, when your code does a *new* operation, and that
would tip the heap over its limit, your app thread gets paused and the
GC is run.

-Ben

--
Ben Hutchison

Lead Developer
Playscape Games
http://www.playscapegames.com

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

captainfreedom
Offline
Joined: 2007-01-10
Points: 0

> I dont think this is correct. Its certaintly not the case in J2SE.
j2ME != J2SE

That code you posted is a bad test, becuase it's impossible to determine if midlet is hung due to infinite loop of just because out of memory. A better test is to print freeMemory on screen before and after assigning memory, and you will see memory stays same. Then try same test, but with a sleep after freeing the memory and you will see that the freeMemory increases. At least that's how gc works on SE phones.

Dhawal Ogale

It's not a bad test case I guess, but the reason it works is that whereas
the GC thread does not get a chance to run on its own, the new operator, by
definition, is guaranteed to NOT throw OutOfMemoryError UNTIL it has
'explored' all possibilities of freeing up memory (including weak references
on J2SE). So I am quite sure this code will run just fine.

So if you create lot of objects in the main game thread, it should not kill
your app, only slow it :-)

-----Original Message-----
From: A mailing list for KVM discussion [mailto:KVM-INTEREST@JAVA.SUN.COM]
On Behalf Of meinterest@MOBILEANDEMBEDDED.ORG
Sent: Monday, May 14, 2007 1:33 AM
To: KVM-INTEREST@JAVA.SUN.COM
Subject: Re: Memory management on "real-world" J2ME devices

> I dont think this is correct. Its certaintly not the case in J2SE.
j2ME != J2SE

That code you posted is a bad test, becuase it's impossible to determine if
midlet is hung due to infinite loop of just because out of memory. A better
test is to print freeMemory on screen before and after assigning memory, and
you will see memory stays same. Then try same test, but with a sleep after
freeing the memory and you will see that the freeMemory increases. At least
that's how gc works on SE phones.
[Message sent by forum member 'captainfreedom' (captainfreedom)]

http://forums.java.net/jive/thread.jspa?messageID=216734

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff KVM-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".