Jump to content


 


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Photo

Help required:Parallel execution on multiple servers in Unix


  • Please log in to reply
8 replies to this topic

#1 sneha1887

sneha1887

  • Members
  • 3 posts
  • OFFLINE
  •  
  • Local time:01:50 AM

Posted 25 April 2016 - 10:57 PM

Hi All,

 

I have a  requirement (in a shell script) to connect to several servers at once and execute a series of commands.

 

I am aware that ssh can be used for sequential execution. But since most of the commands that I need to execute take a long time, I have to go for the parallel option.

 

Is there a way to run commands simultaneously without using any command line tools such as pssh, cluster ssh etc?

 

Can someone please help?



BC AdBot (Login to Remove)

 


#2 NickAu

NickAu

    Bleepin' Fish Doctor


  • Moderator
  • 13,396 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:127.0.0.1 Australia
  • Local time:03:50 PM

Posted 25 April 2016 - 11:28 PM

Post moved from Tutorials to General Linux section by me.



#3 raw

raw

    Bleeping Hacker


  • Members
  • 2,577 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Texas
  • Local time:12:50 AM

Posted 25 April 2016 - 11:46 PM

Hope this helps

https://www.gnu.org/software/parallel/


rawsig.png

 rawcreations.net          @raw_creations


Current systems: WHAT OS, BackTrack-raw, PCLinuxOS, Peppermint OS 6, Kali Linux

and a custom Linux From Scratch server hosting a bunch of top secret stuff.


#4 sneha1887

sneha1887
  • Topic Starter

  • Members
  • 3 posts
  • OFFLINE
  •  
  • Local time:01:50 AM

Posted 26 April 2016 - 12:51 AM

Thanks, Yeah I guess that should work.

So looks like without using a 3rd party tool, this is not possible?



#5 Slurppa

Slurppa

  • Malware Study Hall Senior
  • 646 posts
  • OFFLINE
  •  
  • Gender:Male
  • Local time:08:50 AM

Posted 26 April 2016 - 01:31 AM

Hi sneha

 

I think you could use screen to create multiple session and then in those session connect to your ssh servers.

You can send commands to screen session using -X parameter. For more information you can consult screen manual.


Member of the Bleeping Computer A.I.I. early response team!


#6 DeimosChaos

DeimosChaos

  • BC Advisor
  • 1,420 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:United States, Delaware
  • Local time:01:50 AM

Posted 26 April 2016 - 08:04 AM

If you don't want to use a third party tool, you can get pretty close to simultaneous execution by running ssh then executing the command then back grounding it. I have done something similar to this and works pretty well. If you can have a tiny delay between when the first command executes and the last one does, cause there will be one. So it would be something like this...

ssh user@server1 'command' &
ssh user@server2 'command' &
ssh user@server3 'command' &

Since you background the process, it won't wait until that particular ssh command is executed to move on to the next one. You'll have to play with it, but you can usually do things pretty quick this way. I have written multiple scripts doing similar things to this for quite a few nodes to run something more or less at the same time. Again will be a small delay between first and last, probably less than a second.


Edited by DeimosChaos, 26 April 2016 - 08:05 AM.

OS - Ubuntu 14.04/16.04 & Windows 10
Custom Desktop PC / Lenovo Y580 / Sager NP8258 / Dell XPS 13 (9350)
_____________________________________________________
Bachelor of Science in Computing Security from Drexel University
Security +


#7 raw

raw

    Bleeping Hacker


  • Members
  • 2,577 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Texas
  • Local time:12:50 AM

Posted 26 April 2016 - 08:08 AM

This might also work for you.

https://github.com/k-bx/par


rawsig.png

 rawcreations.net          @raw_creations


Current systems: WHAT OS, BackTrack-raw, PCLinuxOS, Peppermint OS 6, Kali Linux

and a custom Linux From Scratch server hosting a bunch of top secret stuff.


#8 sneha1887

sneha1887
  • Topic Starter

  • Members
  • 3 posts
  • OFFLINE
  •  
  • Local time:01:50 AM

Posted 26 April 2016 - 11:31 AM

Slurppa & DeimosChaos .. Thanks for the detailed explanation.
Will try doing this.
Thanks again.

#9 imort

imort

  • Members
  • 11 posts
  • OFFLINE
  •  
  • Local time:11:50 AM

Posted 14 June 2016 - 07:50 AM

Is there a way to run commands simultaneously without using any command line tools such as pssh, cluster ssh etc?

 

Just run a command like 

ssh user@server1 './long_running_script.sh' &
ssh user@server2 './long_running_script.sh' &

so on.
You'll get a two running SSH instances in the background and the only problem is that you'll not receive any output on the screen with the results.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users