In a SAN Client environment, sometime we will see this errors while the jobs seems running but actually its queuing waiting for the FT Pipes to be opened. But, often the FT pipes are available but some of them has been orphaned due to CORBA exception during cleanup. And if its reached time out you will get error code 83 on netbackup job.
In the Job Details, this will indicates and show you the FT Pipes are in use:
Sep 23, 2011 7:02:09 AM - awaiting resource stug-dpad_nbx_01. Waiting for resources.
Reason: FT pipes are in use, Media server: nbu004.net2.com,
Robot Type(Number): NONE(N/A), Media ID: N/A, Drive Name: N/A,
Volume Pool: NetBackup, Storage Unit: stu-dpad_nbu004_01, Drive Scan Host: N/A,
Disk Pool: dpad_nbu004_01, Disk Volume: /opt/openv/dpad_nbu004_01_06
Now, we list the orphaned pipes:
# /usr/openv/netbackup/bin/admincmd/nbrbutil -listOrphanedPipes
Pipe ID Pipe State Media Server
==============================================================================
3 PIPE_ORPHANED nbu002.net2.com
10 PIPE_ORPHANED nbu005.net2.com
12 PIPE_ORPHANED nbu005.net2.com
13 PIPE_ORPHANED nbu005.net2.com
14 PIPE_ORPHANED nbu005.net2.com
15 PIPE_ORPHANED nbu005.net2.com
16 PIPE_ORPHANED nbu005.net2.com
4 PIPE_ORPHANED nbu006.net2.com
5 PIPE_ORPHANED nbu006.net2.com
6 PIPE_ORPHANED nbu006.net2.com
7 PIPE_ORPHANED nbu006.net2.com
8 PIPE_ORPHANED nbu006.net2.com
9 PIPE_ORPHANED nbu006.net2.com
10 PIPE_ORPHANED nbu006.net2.com
11 PIPE_ORPHANED nbu006.net2.com
12 PIPE_ORPHANED nbu006.net2.com
13 PIPE_ORPHANED nbu006.net2.com
15 PIPE_ORPHANED nbu006.net2.com
16 PIPE_ORPHANED nbu006.net2.com
16 PIPE_ORPHANED nbu008.net2.com
1 PIPE_ORPHANED nbu009.net2.com
3 PIPE_ORPHANED nbu009.net2.com
4 PIPE_ORPHANED nbu009.net2.com
5 PIPE_ORPHANED nbu009.net2.com
6 PIPE_ORPHANED nbu009.net2.com
7 PIPE_ORPHANED nbu009.net2.com
8 PIPE_ORPHANED nbu009.net2.com
9 PIPE_ORPHANED nbu009.net2.com
14 PIPE_ORPHANED nbu009.net2.com
15 PIPE_ORPHANED nbu009.net2.com
16 PIPE_ORPHANED nbu009.net2.com
8 PIPE_ORPHANED nbu002.net2.com
12 PIPE_ORPHANED nbu002.net2.com
1 PIPE_CLIENT_ORPHANED nbu003.net2.com
2 PIPE_CLIENT_ORPHANED nbu003.net2.com
3 PIPE_CLIENT_ORPHANED nbu003.net2.com
4 PIPE_CLIENT_ORPHANED nbu003.net2.com
5 PIPE_CLIENT_ORPHANED nbu003.net2.com
6 PIPE_CLIENT_ORPHANED nbu003.net2.com
7 PIPE_CLIENT_ORPHANED nbu003.net2.com
8 PIPE_CLIENT_ORPHANED nbu003.net2.com
9 PIPE_CLIENT_ORPHANED nbu003.net2.com
10 PIPE_CLIENT_ORPHANED nbu003.net2.com
11 PIPE_CLIENT_ORPHANED nbu003.net2.com
12 PIPE_CLIENT_ORPHANED nbu003.net2.com
13 PIPE_CLIENT_ORPHANED nbu003.net2.com
14 PIPE_CLIENT_ORPHANED nbu003.net2.com
15 PIPE_CLIENT_ORPHANED nbu003.net2.com
16 PIPE_CLIENT_ORPHANED nbu003.net2.com
1 PIPE_CLIENT_ORPHANED nbu004.net2.com
2 PIPE_CLIENT_ORPHANED nbu004.net2.com
3 PIPE_CLIENT_ORPHANED nbu004.net2.com
4 PIPE_CLIENT_ORPHANED nbu004.net2.com
6 PIPE_CLIENT_ORPHANED nbu004.net2.com
7 PIPE_CLIENT_ORPHANED nbu004.net2.com
8 PIPE_CLIENT_ORPHANED nbu004.net2.com
9 PIPE_CLIENT_ORPHANED nbu004.net2.com
10 PIPE_CLIENT_ORPHANED nbu004.net2.com
11 PIPE_CLIENT_ORPHANED nbu004.net2.com
12 PIPE_CLIENT_ORPHANED nbu004.net2.com
13 PIPE_CLIENT_ORPHANED nbu004.net2.com
14 PIPE_CLIENT_ORPHANED nbu004.net2.com
15 PIPE_CLIENT_ORPHANED nbu004.net2.com
16 PIPE_CLIENT_ORPHANED nbu004.net2.com
1 PIPE_CLIENT_ORPHANED nbu005.net2.com
2 PIPE_CLIENT_ORPHANED nbu005.net2.com
3 PIPE_CLIENT_ORPHANED nbu005.net2.com
4 PIPE_CLIENT_ORPHANED nbu005.net2.com
5 PIPE_CLIENT_ORPHANED nbu005.net2.com
6 PIPE_CLIENT_ORPHANED nbu005.net2.com
7 PIPE_CLIENT_ORPHANED nbu005.net2.com
8 PIPE_CLIENT_ORPHANED nbu005.net2.com
9 PIPE_CLIENT_ORPHANED nbu005.net2.com
11 PIPE_CLIENT_ORPHANED nbu005.net2.com
2 PIPE_CLIENT_ORPHANED nbu006.net2.com
1 PIPE_CLIENT_ORPHANED nbu006.net2.com
14 PIPE_CLIENT_ORPHANED nbu006.net2.com
1 PIPE_CLIENT_ORPHANED nbu007.net2.com
2 PIPE_CLIENT_ORPHANED nbu007.net2.com
3 PIPE_CLIENT_ORPHANED nbu007.net2.com
4 PIPE_CLIENT_ORPHANED nbu007.net2.com
5 PIPE_CLIENT_ORPHANED nbu007.net2.com
6 PIPE_CLIENT_ORPHANED nbu007.net2.com
7 PIPE_CLIENT_ORPHANED nbu007.net2.com
8 PIPE_CLIENT_ORPHANED nbu007.net2.com
9 PIPE_CLIENT_ORPHANED nbu007.net2.com
10 PIPE_CLIENT_ORPHANED nbu007.net2.com
11 PIPE_CLIENT_ORPHANED nbu007.net2.com
12 PIPE_CLIENT_ORPHANED nbu007.net2.com
13 PIPE_CLIENT_ORPHANED nbu007.net2.com
14 PIPE_CLIENT_ORPHANED nbu007.net2.com
15 PIPE_CLIENT_ORPHANED nbu007.net2.com
16 PIPE_CLIENT_ORPHANED nbu007.net2.com
1 PIPE_CLIENT_ORPHANED nbu008.net2.com
2 PIPE_CLIENT_ORPHANED nbu008.net2.com
3 PIPE_CLIENT_ORPHANED nbu008.net2.com
4 PIPE_CLIENT_ORPHANED nbu008.net2.com
5 PIPE_CLIENT_ORPHANED nbu008.net2.com
6 PIPE_CLIENT_ORPHANED nbu008.net2.com
7 PIPE_CLIENT_ORPHANED nbu008.net2.com
8 PIPE_CLIENT_ORPHANED nbu008.net2.com
9 PIPE_CLIENT_ORPHANED nbu008.net2.com
10 PIPE_CLIENT_ORPHANED nbu008.net2.com
11 PIPE_CLIENT_ORPHANED nbu008.net2.com
12 PIPE_CLIENT_ORPHANED nbu008.net2.com
13 PIPE_CLIENT_ORPHANED nbu008.net2.com
14 PIPE_CLIENT_ORPHANED nbu008.net2.com
15 PIPE_CLIENT_ORPHANED nbu008.net2.com
6 PIPE_CLIENT_ORPHANED nbu002.net2.com
10 PIPE_CLIENT_ORPHANED nbu002.net2.com
14 PIPE_CLIENT_ORPHANED nbu002.net2.com
16 PIPE_CLIENT_ORPHANED nbu002.net2.com
As you can see, i have a lot of orphaned PIPES to be cleared. Run below command to release the orphaned pipes:
# /usr/openv/netbackup/bin/admincmd/nbrbutil -releaseOrphanedPipes
List back if there's anymore orphaned pipes , it should be gone by now:
# /usr/openv/netbackup/bin/admincmd/nbrbutil -listOrphanedPipes