Tough Case

　　This week I encountered such a case.
　　I was assigned a Very High. Apparently, it is a real Production system down. No user can logon. A food company. But the error code shows it is not very difficult.
　　I performed the first call to customer. Generally it is a information about begin processing and to ask for some bussiness impact for the situation. I used to ask for some log files meanwhile so that I can save another extra round to get them and speedup the processing.
　　This time I met a very hasty person.
　　'Yeah, Yeah, PRD system, sure it is. What shall I do? You tell me. You know, I am very new to this system, you should tell me all the details...'
　　After some talk, I find he is truely very very new. I never expected a PRD system is in a hand of such an administrator and now it is really down and no other person arround. Even worse, they have no connection available. That means, I can't do checks on my own.
　　Well, talk and talk, then I ask for trace log files. I told him to go to the work directory of their system. I tried to tell him the exact path, but I need their System ID, that is the basic information different for each installation.
　　'System ID?' He asked.
　　I almost faint. It is just like he doesn't know his own name.
　　Check the message they reported, I suppose it should 'PRD'. I tried to confirm with him. He still had no idea. I began to lose my patience. My tone changed.
　　But he bumped up first, 'What is your area?'
　　'What?'
　　'Your location.'
　　'Well, I am from China.'
　　'Well, your Singapore colleague never asks so much. They just promise and fix the problem. But YOU asked me to trace. I told you I don't know much of this system. How do you suppose me to trace....'
　　I am shock to death again. And really feel very angry.
　　But the phone is still on, and I must talk. I realize that lose patience is a stupid mistake. Now, I have to fix the situation for that. So after a deep breath, I pull down the tone.
　　'Yes, I understand. Actually I am not to ask YOU to trace, all those trace files are all on your system ALREADY. What you need to do is, just collect them and send them to us for analysis...'
　　He's still in the mode. So I repeat those words for quite a few times before he really get what is the meaning.
　　Finally he calmed down. We get back to the point to find the trace files. My voice is very calm now.
　　'Go to /usr/sap/PRD.'
　　'/usr/sap/PRD....No such command.' He replied.
　　This time I can't bear it again,'Did you type CD SPACE before that?'
　　
　　I can see no hope to solve the problem in this way.
　　Talk and talk.
　　He doesn't know how to transfer files from UNIX to PC and then send to us. This task is performed by his colleague before.
　　Then he asked whether he should restart the system. (So they can afford to stop PRD system to restart it, I think. This is quite different from other cases. Anyway, no user can logon is not far from stop system at all. Maybe they are just a branch site of the retail chain?)
　　I said that might help but I need to check the error first.
　　He decided to restart. (He is interesting. Hasty, hard, pushing, know nothing but very self-confident. Seems more like a admin of that retail bussiness.)
　　Before he put down the phone, I catch him up, 'Do you know how to restart? Not just power off!'
　　'Yes. Yes. My colleague wrote down a detail procedure for me.'
　　'That is to shut down R/3, shut down db, then shutdown machine. Do you know?'
　　'Yes, I know that. You see, we must co-operate. I go to restart the system. You go to check the errors and later tell me how to send you those files. You see, I take the harder part. Emm? Call me back one hour later.'
　　Well, let him go on. I put down the phone.
　　
　　I am really depressed. The quarrel ruins my day.
　　But I have to continue. PRD down is very serious case. Write email to Duty Manager to report the case. In it I emphasize that customer lack experience. Later DM catch me on net and ask for status.
　　'Do you think they need onsite service?'
　　'Yes. I think so.' I replied.
　　.....
　　'What is the cause do you think?'
　　'1.... 2.... 3....'
　　'But why need he restart the system?'
　　I paused for a while, 'He choosed to...'
　　
　　Then someone called him about onsite service. He refused. Nothing is free.
　　
　　I am downhearted during lunch. A tough case.
　　I made mistakes. First, don't show my impatience. Second, should not let him restart, maybe the system will totally messed up in his hand. Third, in the first call I should apply the routine way, just inform the process beginning. Because at that time, I don't have enough time to see the whole picture.
　　Now I undstand why the first line of famous note 'how to handle PRD down' is:
　　'Solution:
　　1. Don't be panic.'
　　This is the only non-technical line in the sea of notes.
　　
　　The later parts are not so bad. An hour later, I called back reluctantly. But this time he only said, 'Not finished, not finished. Give me one more hour.'
　　Another hour later, still not finished.
　　The third hour, he got angry, 'You disrupted me! I can't work well if you call me every hour! Don't call me until tommorrow morning I call you!'
　　When I tell DM all these words, DM can't help laughing. He's really very unique.
　　Then what I can do is just inform him that the service shift is changed to Europe for night and remind him that he can get help from us whenever in problem.
　　
　　Next day, when I called him, he said the system is up and everything is ok. He was very busy and didn't want to talk much about it.
　　DM said the man told us very few, so maybe we need to discuss with him what happened and try to avoid the next occurance.
　　I called again. He still didn't feel like discussion, 'You can close that case now. All work fine.'
　　'So your system is all ok now?'
　　'Yes, everything is ok. We are performing recovery.'
　　So, he did mess it up.

--------------
　Patience, say it easily but hard to hold.
　　Wonderful story. Rejoicing again~~~ pls.
　　
　　So it means the boy lost everything in system?
　　
　　And be prepared the even worse would be, he may complain you didn't give him plenty of service!
　　
　　BTW， do you have any instruction on such cases as if he/she who asks for the help is an absolutely fresh hand?
　　
评论人：linz 评论日期：2004-4-19 11:28


　　Surely not. He may lose something. But since he insisted everything is ok. It means if any loss, it is ok for them. And 'recovery' means, he did something wrong, then he had to recover the things. That does not equal to 'loss'.
　　
　　Well, he can complain. That is his right. But, the point is, apparently it is not my fault. Generally I stick to the procedure. I provided the right solution in all records. However, whether he acted per my suggestion depends on him.
　　(It is always safe to stick to procedure... :)) That is very important. It is not some individual person or some superman that garantees everything work in the right way. Only wholesome procedure/rules preclude possible risk.）
　　
　　About the fresh hand:
　　PRD system is not for play/ for taining / or for experiment. Every minute/hour system down means money losing. The scale depends on your bussiness scale, millions of dollars or nothing. You can think in this way: suppose your company's manufacture line loses power for some time... PRD system is directly connected to your manufactory or sales or financial etc. functional departments.
　　So for such a urgent case, we provide promptly suggestions. If they setup private connections correctly, we logon their system, check and fix the things for them.
　　If still the man can't handle in this way, we provide onsite service--surely it charges.
　　So the problem for a company is: hire a qualified administrator, train this man correctly, pay the onsite service fee or take the risk of bussiness loss. Just choose the most in-expensive one.
　　(Well, this part is toooo bussiness-like tone. Forget about it. Anyway, it is the fact and how it is working)
　　
　　From my side, I am still not composed enough. I lack the experience to control the situation better. This part is what I feel bad. It needs effort and knowledge.
评论人：乱看 评论日期：2004-4-19 15:30