In short, don't rely on the finalize() method as your only cleanup method, especially if it is deleting files. Instead make up a cleanup method (cleanup() works nicely) that lazy deletes things. Here is an example.
public class Example {
boolean isCleanedUp = false;
public synchronized void cleanup() {
if (isCleanedUp) return;
// flag as cleaned up
isCleanedUp = true;
//...