Jump to content


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.

Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.


How to check multiple url exists or not using the unix scripting?

  • Please log in to reply
2 replies to this topic

#1 sathyaraj


  • Members
  • 1 posts
  • Local time:12:43 PM

Posted 08 March 2016 - 01:55 AM

Hi All,
I am new to unix , i want to check the url exists or not and url status .
kindly help me how to do it 
i am using the below one for checking the single url..
curl -s --head http://google.com/  | head -n 1
i want to check multiple url in single execution..
i want to get the status report like below when checking the multiple urls
google.com status code 200 and url exsists
yahoolzaa.com status code 400 and url doesnot exist
gmail.com status code 200 and url exists
kindly help me to get the output with status report pls..

BC AdBot (Login to Remove)


#2 Guest_hollowface_*


  • Guests

Posted 09 March 2016 - 03:59 PM

I'm not sure if you're requesting a script, or seeking help writing your own script? It's also unclear to me which OS you are using. If it really is Unix, then I cannot help, as I've never used that OS. I have used some unix-like ones, but most of my experience is with Linux operating systems.

If you'd prefer to write your own script, you might find the chosen answer from ( http://stackoverflow.com/a/25136723 ) helpful as a starting point.

If you are seeking a pre-built script, I've written for Linux, a url checking script called Obituary, which can be found ( http://www.bleepingcomputer.com/forums/t/607572/obituary/ ). However it doesn't output in the format you want, and may not work on your OS. I've made an alternative version which (assuming I've understood what you want) does output in the requested format. It's written for Bash, so you must use Bash or a compatible shell. I'm running Ubuntu 14.04 Desktop AMD64 (a Linux operating system), so that's what it has been tested against. It will report 200 as exists, 400 and 404 as doesnot exist, and anything else as Undetermined. You can change that by editing the if elif else statements at the bottom of the "obituary-checker-task" file.

Steps To Install & Use The Alternative Script:
1. Download the ZIP archive from "https://mega.nz/#!M9gVCSID!oVycTeDiZI3Rpfd3X6ZCPHJfx9L14FE1rhTv8PduOjw". If you want to verify the checksum the SHA-512 is 88e49d1c18a2706e75bee4bdf883effe3853da69d758245cf23cea5bb0d9eee66c38a86a65664587dc8d9b9742d86acb215a26521d1a9af9031fbe0d84c96a42
2. Install tee, bash, sed, mawk, and curl. If you aren't sure how to install these programs, state which OS you're using, and I (or another user) may be able to help with that if we're familiar with the OS. Like I said, I'm a Linux user.
3. Extract the ZIP archive to a folder. I'd suggest to a folder named obituary, located in your user's home folder.
4. To to use type:

bash /directory/obituary-cli --location="/directory" --urllist="/directory/file" --output="/directory2/file"

You'll need to substitute in the correct paths and filenames for this command to work. Location refers to the folder where the script is. Urllist refers to the location of a plain-text file containing the urls you wish to check (one per line), and output refers to a file where you want to save the results of the check. You must use the fullpath for all of these, you cannot use things like "~/".

Screenshot of the script in action:

Edited by hollowface, 09 March 2016 - 04:02 PM.

#3 Captain_Chicken


  • BC Advisor
  • 1,369 posts
  • Gender:Male

Posted 21 March 2016 - 07:31 AM

Unix isn't an operating system. It is a codebase that Linux is based off of. Are you referring to bash, the terminal language/scripting?

Edited by Captain_Chicken, 21 March 2016 - 10:11 AM.

Computer Collection:





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users