fool proof file transfer

[dev]

hello all,

how can we provide reliable file transfer?, i mean solutions to problems that can happen on a long file transfer, say server goes down, or client goes down, what i really want is to have a file transfer , where by if the file is transfered , it should be complete else nothing should be there with reciver, above all the client should not be able to say that he didnot recive the file,if the transfer was sucessfully completed.

i want a system where by after the file is fully transferd to the client, the client application should acknowladge(the application,i mean). but the problems will arise, if for example:-
- the file is fully transfered to client, now he has to send the acknowladegment, but the power goes at client side, now that he has the file , but has not acknowladged. what can be solution to these type of situations?
if i keep logs , still the problem will be there, like what happens if at time of writing of log ,power goes. i think all these are the problems related to syncronisation and related to transation processing, like that of a database system, though we have been taught bout these probs in classes , but no solution was discussed.

so what do u guys suggest?
with thanks
-dev

[RobSeace]

My suggestion? Relax... ;-) Yes, it's interesting to consider
all sorts of pathalogical cases and horrible conditions that
might arise... But, in reality, you'll probably never encounter
them, and you'll just drive yourself mad if you seriously try to
robustly handle each and every one of them... ;-) No, that's
NOT to say you should just give up, and design shoddy
software that doesn't worry about handling bizarre conditions...
(Though, it seems a lot of people have chosen that route... ;-))
I'm just saying, don't think too hard... Work around the
obvious/easy problems, but if you find yourself up against a
difficult problem, step back and consider the possibility that
it may simply be unsolvable, period... You may just have to live
with the fact that there's no perfect solution, and the best you
can do is reduce the possibility of problems arising as best you
can, and/or reduce the impact of any problems should they
actually arise... Eg: in your case, there's really NO way at all
to ever be certain of having both ends in perfect sync on the
condition of that transfered file... No matter what you do, you're
always going to have the possibility of either the receiver side
thinking its got a valid file but being unable to convey that to the
sending side, OR the sending side thinking it successfully sent
the file before the receiving side truly had it successfully stored...
So, first, just realize which condition is least worrisome... In
most cases, it would be the former... What's the worst that could
happen in that case? Maybe the sender will try to re-send the
same file again later in the future? Just allow for that possibility
in your design, and try to deal with it, somehow... Maybe it won't
do any harm to have the same file re-sent in the future? If so,
no problem, nothing to worry about... But, if it's some file that
needs to be processed, and processed only ONCE, then you
have to be a bit more careful... Idealy, you'd want some method
of recognizing the fact that you already processed this file before,
if it should show up again in the future... Maybe you keep track
of the md5sums of all files processed, or something like that, and
check all new files against the previously processed list... Or,
maybe each file has some unique identifying marker in it, which
you can use to keep track of processing... *shrug* But, in
general, I think that's the sort of approach you need to take in a
case like this: limit failure conditions as much as possible, but
recognize the fact that you simply CAN'T get rid of them entirely,
so plan around the fact that things may still fail, and limit the
damage caused in such a case...

[dev]

hi all,

thanks, well i too agree that i will be mad if i try too hard, but still i find this to be really intresting. i am just a beginer, so i have to take things slowly. any way thanks, i am not getting dishearted or giving up, u guys r really helpfull. what is life without complexities?

-dev