Skip to main content

How to stop EJB timers being deleted upon app undeployment

13 replies [Last post]
mrawl
Offline
Joined: 2004-06-27

Hi, is there way to stop EJB timers from being deleted when an app is undeployed? I've tried using a separate data source, but it still deletes them. This is pretty drastic behaviour given that none of the other app data is deleted (which is what I want, based on configuration of the persistence unit). I understand this behaviour used to be different but was switched - when it was switched, was some sort of configuration put in place to enable the old behaviour? Please don't respond with "but you're undeploying, it makes sense " - no, it doesn't make sense in many cases... Thanks :)

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
bebbo
Offline
Joined: 2009-03-23

As many pointed out correctly:

  • The timers are tied to your application.
  • Undeploying the application must and will delete the contained timers

The solution is easy:

Create a separate application for your timers. That timer application can access the main application as any client and do whatever you need.

No hacks, squirks, ...

whartung
Offline
Joined: 2003-06-13

> calls the appropriate handler
> methods which in turn can switch Subjects to work
> with JAAS, depending on whether it is a system or
> user timer.

I'm curious how you go about doing this. We do a similar thing, but we use Programmatic Logic, and an ugly, ugly, hack.

mrawl
Offline
Joined: 2004-06-27

Actually, the subject is switched before calling the handler (so it doesn't have to worry about it, and is just a normal bean method) - the timer holds the user info.
For example, this executes the handler as either the given user or anonymous...

if (emailAddress == null)
AnonymousUser userPrincipal = new AnonymousUser("$anon$");
else
AuthenticatedUser userPrincipal = new AuthenticatedUser(emailAddress);
Set principals = new HashSet();
principals.add(userPrincipal);
subject = new Subject(false, principals, new TreeSet(), new TreeSet());
Subject.doAsPrivileged(subject,
new PrivilegedExceptionAction() {
public Object run() throws IOException, ServletException {
handler(args);
return null;
}
}, null);

mrawl
Offline
Joined: 2004-06-27

Hi, thanks for the great answers guys. I really appreciate that.

Re: always been like this - other app servers, eg, JBoss have a setting to prevent timer removal. They occasionally refer to the automatic removal of timers as "the old way"...

Re: timers belong to app, not container. Yes indeed - that is **precisely** my point. And yes, indeed, timers are not like other shared resources, they are much more like app data resources. Which is why they should be under control of the app, not the container. And just like my app can be configured not to drop data tables, same should go for timers.

So let me throw out a strawman RFE - allow the timers to be placed under a persistence unit. Maybe move/add some timer apis to the entity manager or sim. These timers would have a PU based life cycle. Question - if this were done, would there be any use for the current timers? I'm not sure, perhaps they could coexist.

I agree that leaving them on undeploy is a problem given the current setup. I believe the design/spec has problems here.

I thought of all those things you mentioned, in the end I simply wrote my own timers and stuck them in my own db. I'm using foreign keys instead of handles, which gives me better integrity. (at the loss of generality).

I'm using a servlet context listener to start (and end) my timer thread. I have control over transaction batching, timer semantics (like repeat every month, etc), granularity. I did this because I could have 100,000s of timers and I cannot trust the container implementation to scale, and I don't want to be restoring them on redeploy. So, I'm happy, but also sad that one of the main reasons I chose EJB technology turned out to be moot.

~Mark

ksak
Offline
Joined: 2005-05-20

Hi Mark,

Yes, some vendors do preserve timer data across deployments. We just never have in SUN's timer implementation. The topic has come up on the EJB 3.1 Expert Group but it's unlikely to be standardized. To some extent, the proposed EJB 3.1 timer facilities mitigate the issue since there will be support for automatic creation of timers based on a cron-like schedule.

Performance-wise, the database overhead of persistent timers is likely to dominate that kind of scale out. I haven't done any specific tuning exercises for very large timer apps but I'd be interested in hearing any feedback you have on your results. We're also planning to introduce a non-persistent timer option in EJB 3.1. It has a few benefits but one is reduced overhead. That might be a better fit for fine-grained timers and/or very large numbers of timers.

--ken

mrawl
Offline
Joined: 2004-06-27

Thanks Ken,

Just to clarify something, the reason I wanted the timers preserved is because I find myself having to "undeploy and redeploy" fairly often (typically from Netbeans). This is partly because Netbeans is somewhat unreliable in the way it deploys. I've since found a way to make netbeans behave a bit better, but still, I believe the life cycle and mgt of timers should definitely be consistent with other app data, especially since they are tied to EJBs - they should live and die strictly with those EJBs, therefore under control of the persistence unit. As to scalability, I don't know what to expect from the container timers, but their granularity is certainly much finer than I want - I only need to check every 15 minutes or so. Anyway, I don't want to risk having to temporarily undeploy again some day, or if I want to move my app to a different server, etc - so I have to control the data in a single db with consistent life cycle. If my timers don't scale I might reconsider, but I expect they will scale very well. I'll let you know if I come up with any interesting stats on this.

Mark

whartung
Offline
Joined: 2003-06-13

Yea, the timers do kind of exist in a "no mans" land. On the one hand being applications state (and not just application state, but persistent applications state), but on the other being a container artifact that the container is somewhat responsible for (thus dumping them on undeploy).

I think it would be a fair RFE to make Timers persistent across redeploys as a deployment (or, perhaps, undeployment) option.

As for the scaling issue, that 100,000 is certainly a big number, and I can easily see an implementation being blindsided by such a volume of timers.

On the one hand, the naive implementation would be to simply load all of the timers in memory at start up. For any "reasonable" number (< 1000, say, depending on the size of the getInfo blob), that's not an absurd idea. Frankly, it's not even an absurd assumption.

But when that number goes to 100,000 -- yea, there could be issues.

As an alternative, it could be implemented as a simple query:

SELECT * FROM TIMER WHERE RUNDATE <=

And just run that every minute, queuing up each job. That will scale until the cows come home. Now, if you're firing 1000's of jobs per minute, well, there are other issues involved with that as well.

That works well and good until someone calls "getTimers()". Then, there's, well, ouch. That call is not scalable.

Frankly I think your solution of managing your own timers, giving your volume, is probably best. I would have most likely leveraged the timer service as the runtime engine for such a model (rather than manage my own threads), plus the timers jobs run in the EJB tier, whereas servlet threads don't. Yea, it's kind of a moot point if all your servlet thread does is call a session bean, but, still. I like that the timers are in the EJB tier. Suits my world view better.

mrawl
Offline
Joined: 2004-06-27

Yes, correct - I use a simple query like that and it will scale massively. The timer thread periodically calls timerBean.fireTimers() which has a maxTimers property to control the maximum size of the transaction - I'll fire 10 timers per transaction as a first guess. fireTimers() then does a simple indexed retrieval and calls the appropriate handler methods which in turn can switch Subjects to work with JAAS, depending on whether it is a system or user timer. I have little doubt that this is more scalable and more manageable than EJB timers. My timer web service is switchable between the two types though, so I can easily try them both out and see what happens. I agree it should all happen in the EJB layer, but as you know the thread can't really be launched there, so apart from that one hiccup, it does indeed all happen in EJB land. Thanks for the very valuable input, it's good to toss these things around a bit once in a while.

mrawl
Offline
Joined: 2004-06-27

I should point out that I could use a single EJB timer in place of my timer thread, but even then the problem of (un)deployment is a show stopper for me - I just don't want to deal with that design.

mvatkina
Offline
Joined: 2005-04-04

As there are currently no support to create timers on deploy, your application must have some business logic that creates the timers. What is the problem with executing this logic after redeploy?

thanks,
-marina

mrawl
Offline
Joined: 2004-06-27

The timers I'm using are just entries in my db. I don't need to (re)create them on restart, redeploy, etc, because they are there already (in the db). The only thing I need to do is start my timer thread. If I was using EJBTimers, then after undeploy/redeploy I would need to recreate them because unlike the rest of my app data they would have been unconditionally removed - that is precisely what I'm trying to avoid. Simply put, I want to own all of my data, in one place, with one set of semantics. Again, it's true that things are fine if I simply never undeploy, but I don't like that hanging over my head, for the reasons I've mentioned, including moving my app to a different server, testing, etc.

whartung
Offline
Joined: 2003-06-13

Well, I'm going to reply anyway.

It DOES make sense, because EJB timers are specific to the APPLICATION, not to the CONTAINER.

All of those other resources that your app relies upon are shareable among all of the applications within the container. EJB Timers are not. EJB Timers are tick down timers tied to a specific Session Bean. When the Session Bean goes away, there's nothing to tie the timer too. Now, certainly, the container can just sit there and silently skip or spam your logs about timers missing session beans. But, really, since the timer is an application artifact, it really should be cleared when the application is undeployed.

Now, is this a pain point? It certainly can be. I think that the timers should be more accessible, from an operations point of view, than they are now.

At the moment, the solution is simple, but lightly hackish. Since the EJB tier doesn't have any real accessible life cycle, you need to bundle a WAR with your EJBs, and that WAR at a minimum will contain a Servlet that loads on start up, and in its init() method, it calls the App tier to reconfigure the timers (typically a method in the timer bean itself). Make sure you don't just insert the timer twice, need to check that it exists first, as your method will fire not just on deployment, but anytime the container starts up. (See the TimerService.getCurrentTimers() method to view the state of the timers in the system, and put something relevant in the Timer.info property to identify your timers.)

Our app can have any number of arbitrarily configured jobs, and we store them in the DB. On start up, we read that table and recreate the timers so they fire properly.

As nice as it would be to just "fire and forget" the timer jobs, it brings up all sorts of issues of how to handle restarts and downtime, etc.

For simple, regular jobs, the Timer is a fine service. As the demands grow, it turns in to a really base asynchronous execution mechanism that needs to be augmented with application code. They could pump up the features some, but it's really one of those services that kind of either works for you or it doesn't, no matter how many features the pour in to it. Some folks want it for "cron", others want it for just wrong background tasks (I use it for both).

As painful as maintaining timers may seem, I think there's a hidden benefit to having your application having complete control over the jobs there.

Anyway, do file an RFE for what you would like to see (the more detailed the better), maybe something can be done in the spec and if not, perhaps in v3.

ksak
Offline
Joined: 2005-05-20

Glassfish doesn't provide a way to preserve EJB timers after undeployment. This has always been the behavior. It's unrelated to the auto creation/deletion behavior of application tables for applications containing CMP 2.x entity beans and java persistence entities. Your best bet is to file an RFE but there is no current workaround.