Quesiton about synchronization

[TheWhite]

If I have 2 synchronized methods and 1 calls the other, what exactly happens?

If those 2 synchronized methods are in a class, and 2 of those classes are created, each of them calling one of their synchronized methods, what would happen?

If those 2 classes extended Thread and were started and each called one of their synchronized methods, what would happens?

After reading the java documentation on synchronization I'm still a little confused...... Are synchronized methods only useful when multiple threads are using the methods? And in regards to the 2nd and 3rd questions, would the calls on the objects methods serialize, having 1 wait for the other to finish with the synchronized function, before it calls it next? I'm just trying to better understand where I would need to use synchronization.

My situation is this: I have a FTP class, and an FXPThread class which uses 2 FTPs instances in order to transfer (FXP) between them. When I try to run a single FXPThread, the transfer works fine. When I try to add a 2nd FXPThread to run simultaneously, both connect fine, but randomly, only one of the threads simply hangs on one of the transfers. I figure there has to be some kind of synchronization issue or some kind of java resource issue with sockets or something, but I cant seem to figure it out...

[Andreas Hollman]

Hi,
synchronized method

class MyClass{
 public void synchronized myMethod(){
     //your code
 }
}

is the same as

class MyClass{
 public void myMethod(){
     synchronized(this){
         //your code
     }
 }
}

:-)
Andrej

[TheWhite]

Thanks, Andreas but that doesnt really explain what changes, synchronizing does to the actual methods when the program goes live. I already know that much :)

I mean, I saw this in the java documentation about deadlock:

public class Deadlock {
    static class Friend {
        private final String name;
        public Friend(String name) {
            this.name = name;
        }
        public String getName() {
            return this.name;
        }
        public synchronized void bow(Friend bower) {
            System.out.format("%s: %s has bowed to me!%n", 
                    this.name, bower.getName());
            bower.bowBack(this);
        }
        public synchronized void bowBack(Friend bower) {
            System.out.format("%s: %s has bowed back to me!%n",
                    this.name, bower.getName());
        }
    }

    public static void main(String[] args) {
        final Friend alphonse = new Friend("Alphonse");
        final Friend gaston = new Friend("Gaston");
        new Thread(new Runnable() {
            public void run() { alphonse.bow(gaston); }
        }).start();
        new Thread(new Runnable() {
            public void run() { gaston.bow(alphonse); }
        }).start();
    }
}

I'm still unsure how making them synchronized creates the deadlock, so I asked the above 3 questions to try and figure out the difference. What I really need are answers to the above questions.

[Andreas Hollman]

1)

If I have 2 synchronized methods and 1 calls the other, what exactly happens?

you lock the same variable (this) two times:

synchronized(this){
   synchronized(this){

  }
}

2)

If those 2 synchronized methods are in a class, and 2 of those classes are created, each of them calling one of their synchronized methods, what would happen?

MyClass obj1 = new MyClass();
MyClass obj2 = new MyClass();
obj1.myMethod(obj2);

obj1.myMethod(...) is synchronized and you call a another synchronized method of obj2, then it is the same as the code below, here you can get a deadlock!

synchronized(obj1){
   synchronized(obj2){

  }
}

(3)

If those 2 classes extended Thread and were started and each called one of their synchronized methods, what would happens?

They are extended from the thread, but they are normal java-instances and can be synchronized in ussual way like any java-object. Thus if you call any synchronized method inside of run method it is allmost the same as:

synchronized(this){
//code inside of synchronized method
}

To avoid deadlocks you can use optimistic aproach like transactional memory or pisimistic aproach like multi-isolation

:-)
Andrej

[TheWhite]

Ok, great this helped.

Can you explain why this deadlocks?

synchronized(obj1){
   synchronized(obj2){

  }
}

[Andreas Hollman]

Ok, great this helped.

Can you explain why this deadlocks?

synchronized(obj1){
   synchronized(obj2){

  }
}

you have two threads and they can lock this objects in order:
Thread1 locks obj1, Thread2 locks obj2 but Thread1 can't lock obj2 and Thread2 can't lock obj1. Threads waits foreach other.

synchronized(obj1){
   synchronized(obj2){

  }
}

synchronized(obj2){
   synchronized(obj1){

  }
}

wiki

:-)
Andrej

[TheWhite]

ok, so I have a question...

If I synchronize the mkdir and chdir functions in my FTP class and I have 2 FTP instances and one calls mkdir and the other calls chdir, do the calls synchronize so that 1 ftp mks the dir and the other can enter the same dir without the 2n'd throwing a "does not exist" exception?

I'm pretty sure this is why I would synchronize the functions. I just want to make sure ;)

[JamesCherrill]

You should create a variable just to use as a synch object then if two pieces of code should not run concurrently, explicity synch them on that same variable. If there are two different situations where that applies you can have two variables (etc). Avoid having to synch on two variables simultaneously (avoids deadlocks). Keep it simple, keep it explicit. eg

Object dirLock = new Object(); // just for synching
...
synchronized(dirLock) {
   // mkdir code
}
...
synchronized(dirLock) {
   // chdir code
}

just don't call either fo these pieces of code from within the other!

[TheWhite]

Let me make sure I'm understanding this correctly. So really, the synchronized functions wouldn't help me do what I'm trying to do since:
synchronized (this) only takes ownership of the lock for (this).

I would have to create a generic static object and synchronize (the object), so only 1 thread is able to lock it at any time.

Am I right with this?

[JamesCherrill]

Yes.

[Andreas Hollman]

Let me make sure I'm understanding this correctly. So really, the synchronized functions wouldn't help me do what I'm trying to do since:
synchronized (this) only takes ownership of the lock for (this).

I would have to create a generic static object and synchronize (the object), so only 1 thread is able to lock it at any time.

Am I right with this?

The method which use one object for global synchronization is called "global locking" :-)

[~s.o.s~]

IMO, the operations(mkdir & chdir) should be part of the same thread of execution. Also, creating a thread-aware Ftp class doesn't seem like a good idea unless of course you provide a provision to schedule multiple FTP actions synchronously.

I would have to create a generic static object and synchronize (the object), so only 1 thread is able to lock it at any time.

*Don't* use static variables for synchronization, at least not in your case. They seriously limit the way in which a class can be used and usually turn out to be scalability killers.

In your case, you need to ensure that both the FTP thread classes need to synchronize on a logical instance (and not a static variable). You can for e.g. create a FtpWorkflow class which would be passed to your Ftp thread class during creation. This workflow class can be used to share information between two Ftp operations along with providing a common ground for synchronizing the FTP operations involved in the workflow.

You can take this concept a bit ahead and expand on it by making FtpWorkFlow implement Runnable. This workflow class would now support scheduling of Ftp commands and make sure they are executed synchronously thereby hiding all the boiler-plate scheduling code.

You can of course use any XXX class for the same with a functionality of your choice though the basic idea remains the same.

[TheWhite]

Very good.

Actually, the reason I need to synchronize the mkdir and chdir functions between 2 FTPs is because I am FXPing (xfering) between them.

I have an FXPThread class which has the 2 FTP instances and fxps the list of files from the source ftp to the dest ftp (this kinda works like your workflow class). The problem is, when 2 FXPs happen at the same time for the same folder (for example CD1 and CD2 of folder), folder is created then entered by one, then attempted to be created a 2nd time by the other, throwing a "already exists" error

As per your idea, I could have had the FXPThread pass itself (this) as the dirLock object to each of it's FTP constructors instead of using a static object, but that wouldn't help my situation. My situation is when 2 different FXPThread FTPs try to create the same dir at the same time. The only thing I can think of is to use a static object global to all FTPs to lock on, or to pass the same object to all of them, not per FXP, but for each FTP in general. In that case, a static dirLock object in the FTP class would work best, no?

[~s.o.s~]

folder is created then entered by one, then attempted to be created a 2nd time by the other, throwing a "already exists" error

If this is the case, then I think you are over-engineering the problem. A simple existence check for the directory is much more logical and simpler than trying to synchronize `n' FTP instances. Have you tried that out yet?

The only thing I can think of is to use a static object global to all FTPs to lock on, or to pass the same object to all of them, not per FXP, but for each FTP in general

Then you are basically making all your FTP actions/calls execute in sequence or only one FTP action at a given time. Your FXPing class uses FTP classes to do the real work. If you sync on a global static object and that too in your FTP class, it's as good as not using threading approach, at all.

[TheWhite]

folder is created then entered by one, then attempted to be created a 2nd time by the other, throwing a "already exists" error

If this is the case, then I think you are over-engineering the problem. A simple existence check for the directory is much more logical and simpler than trying to synchronize `n' FTP instances. Have you tried that out yet?

It is doing an "existence check" before creating the directory. The problem is, 2 threads are doing the same exact check at the same exact time.

FXPThread1:
Check Fails
Create

FXPThread2:
Check Fails
Create (oops, already exists cause FXPThread1 created it)

This is why I need synchronization between FXPThreads by a common locking method and why the static dirLock object kinda works.

synchronized(dirLock) {
if (check fails)
create
}

The only thing I can think of is to use a static object global to all FTPs to lock on, or to pass the same object to all of them, not per FXP, but for each FTP in general

Then you are basically making all your FTP actions/calls execute in sequence or only one FTP action at a given time. Your FXPing class uses FTP classes to do the real work. If you sync on a global static object and that too in your FTP class, it's as good as not using threading approach, at all.

I see what you're saying. But, if all FTPs are forced to lock on this static dirLock object only when folders are being entered or created, the delay should be fairly minimal -- it will just serialize every ftp's mkdir and chdir combination (lock, release, let the other one go, lock, release, etc), instead of having them do the check and create asynchronously where a problem could happen.

Granted, a lot of the folder changes and creations do not require this synchronization so I would like to avoid it wherever possible, but I just can't think of any easy way.

This is a pretty interesting bottleneck so I'm all ears :). The complexity of my program requires independent FXPs to happen, untimed, and unorganized so the above situation could very well happen (and it has).

To give you an example of how complex this program is, an FXPThreadQueue is also implemented, which adds FXPThreads and runs them based on a MAX_THREADS variable and queues the rest until a spot opens so flexibility is a huge deal. Two FXPThreads could very well be working on the same folder (folder/CD1 & folder/CD2) so folder/ may be chk/mk at the same time by both threads.

[JamesCherrill]

If you have a variable representing the FTP server for any given operation, maybe you could synchronize the dir-making stuff on that object, thus only causing a thread wait when it really is necessary?
Or
Perhaps you could have a small class that just manages lock objects based on the fully-qualified directory name
public static synchronized Object getLockFor Directory(String directoryName)
{ have a hashmap<String><Object> of dir names and lock objects
return existing or new object as appropriate }

[~s.o.s~]

You can try out the following things:

• Place the folder creation code in the try...catch block and catch the specific FolderExistsException exception. When this exception is caught you know for sure that the target folder exists so you need not create it. You can just ignore this exception and continue with your work.
• You can try the directory name to object mapping approach for locking as already suggested by James
• You can have a workflow class as mentioned in one of my previous posts, but this time for your FXPThread classes. This way you can explicitly handle the dependencies.

[TheWhite]

If you have a variable representing the FTP server for any given operation, maybe you could synchronize the dir-making stuff on that object, thus only causing a thread wait when it really is necessary?

I'm not sure what you mean by synchronizing the dir-making stuff on "that object" --- right now all dir-making stuff is synchronized on the static lockDir object of the FTP class.

Would the try...catch work? The only way it will work is if it does synchronizing against all threads. Remember, the problem here is that 2 threads are calling the same check-create code, at the same time and 1's create may change the state in the middle of the other's check-create statement.

Workflow class wont work either. There could be up to 5 FXPs at a time, happening simultaneously, with possibly all 5 working on the same dir at the same time, creating and changing, unknowing in what order and by who and when -- cant do explicit handling that way.

The hashtable sounds interesting, but implementing it wont solve the problem, it would just make it worse.
__
I can have a static hashtable for the FTP class (instead of a single static object) and add to it every time a mkdir or chdir is called, keying it to the folder's name and a new Object(). If the folder is already in the hashtable, use that object.
__
There is 1 issue with that. The hashlist would have to be synchronized so that if a folder is added while also being searched for, it will be found. Here, begins the problem --- since only one instance can search/modify the list at the same time, we in fact did nothing except slow it down even more by adding searching....

getHashedLock(folder):
locate folder's dirLock object in hashlist
if found, use object //blocks hashlist while searching
else create new key //blocks hashlist while adding

FTP1:
synchronized (gethashLock(folder)) //blocks hashlist, then blocks the correct object
mkdir or chdir

FTP2:
synchronized (gethashLock(folder)) //blocks hashlist, then blocks the correct object
mkdir or chdir

the hashlist is still blocked for both ftps, so every ftp still has to wait for it to be unlocked.

The ideas fail.

[~s.o.s~]

Would the try...catch work? The only way it will work is if it does synchronizing against all threads. Remember, the problem here is that 2 threads are calling the same check-create code, at the same time and 1's create may change the state in the middle of the other's check-create statement.

When you rely on gobbling exceptions + the correctness of the FTP library you are using, you need not bother with the 'check' phase. The code would be something like:

ftp.connect();
try {
    ftp.createDir("myDir");
} catch(DirAlreadyExistsException e) {
    // dir already exists, use it
}
ftp.changeDir("myDir");
ftp.storeFile("myFile", myFileIoStream);

This *should* work; if it doesn't, something is seriously broken.

[JamesCherrill]

The point about using the hashtable is that the synched method to add/retrieve the directory-specific lock object from the hashtable will take microseconds (the hashtable is only locked for the time it takes to check/update the hashtable then it is released), and there will be zero unnecessary locking of the FTP activities. The hashtable itself will not be locked during an FTP operation.

[TheWhite]

Using the hashtable would work, but it would force a lock twice. Once to search or add to the hashlist, then once again on its object. Maybe it will be faster if there were 100 FXPs and half of them were messing with the same folder. Maybe its something to look at.

~s.o.s~'s idea of making the mkdir function throw an exception if the dir is already there is also plausible and may actually be better for this situation and avoids synchronization all together (I don't know why I was making it harder than it was).

Either way, I think I learned a slightly better way of doing things and also cleared most of my issue with understanding synchronization. I would like to learn more of exactly what happens to that object we are locking on though, but thats for a later date.

Thanks to you both. :)