Jump to content


 


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Cloudy


  • Please log in to reply
33 replies to this topic

#1 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 08 November 2015 - 04:10 PM

HSeuXxS.png

While I was listening to music yesterday evening I decided to do some fiddling in an Ubuntu VM, just to give my hands had something to do. I ended up writing a small bash script called "Cloudy" that works as a cloud-scanner for malware. Cloud meaning it has no locally stored definitions. Basically, the way it works is you tell Cloudy what file to check, it hashes the file using SHA-256, searches for that checksum on VirusTotal.com, and outputs the detection ratio listed on VirusTotal.com, which allows the user to decide if they want to trust the file. As you can see it's very basic, and not overly useful, but considering it was thrown together in an evening it's kinda cool. It should work on a variety of distros, provided you install it's dependencies, but I built it on Ubuntu 14.04 Desktop 64bit, so that's all I've tested it with. This is a testing release, so it could have bugs. I have no plans to continue working on it or testing it at this time. Anyways, I thought I'd share the download link for anyone whom is interested.
 
Install Cloudy 0.17 Testing:
1. Download the text file from:

2. Copy the code between the

[quote][/quote]
sections into a new file, and save as "cloudy". It doesn't matter where you save it. I saved mine in my user's home folder (eg: /home/example1/cloudy).
3. Install the following dependencies: Sed, Lynx, and Coreutils. The manner in which you do this will vary from distro to distro. Both Sed and Coreutils are pre-installed on Ubuntu. On Ubuntu and Ubuntu derivatives these can all be installed by typing In terminal:
sudo apt-get install sed coreutils lynx
 
Run Cloudy:
1. In terminal type:
bash /directory/cloudy
(Substitute "/directory/" for the path to where the "Cloudy" file is saved.)
 
Not all files are listed on VirusTotal.com, obviously. In that case you'll get an output saying the file couldn't be found in the database. If the file is found you'll get something like # / #. The first number represents the number of virus scanners that detected the file as dangerous, the second number represents the number of scanners that the file has been scanned with. Ideally you want to see 0 / #.

I submitted a copy of Cloudy (the actual script, not the text file) to VirusTotal.com, you can view that here, or scan Cloudy with itself :P.

Edited by Al1000, 03 September 2016 - 12:25 AM.
delete download link


BC AdBot (Login to Remove)

 


#2 Al1000

Al1000

  • Global Moderator
  • 7,597 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Scotland

Posted 09 November 2015 - 01:57 AM

Very neat! :)

#3 NickAu

NickAu

    Bleepin' Fish Doctor


  • Moderator
  • 13,103 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:127.0.0.1 Australia
  • Local time:01:46 PM

Posted 09 November 2015 - 03:56 AM

See now you going to make me install it.


Arch Linux .
 
 Come join the fun, chat to Bleeping computer members and staff in real time on Discord.
 
The BleepingComputer Official Discord Chat Server!


#4 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 09 November 2015 - 06:16 PM

I know I said that I had no plans to continue working on this, and it's true, but I had a lightbulb moment so I did. As said before, while this should work on a variety of distros I've only tested it on Ubuntu 14.04 Desktop X86-64, and as before this is a testing release, so there could be bugs. So if it works for you that's great, but if it doesn't there's nothing I can do about it. Just posting it for anyone who's interested.
 
This release supports scanning individual files, or recursively scanning all the files in a directory. By default it still outputs results to the terminal, but you can also optionally save a copy to a file. I still have no plans to continue working on or testing this, but if I do the next logical step would be to expand cloudy into a set of scripts so that I can add an OPTIONAL file deletion feature. The idea being that the user can delete files based on their detection ratio. We'll see.
 
Example Output:
 
example1@example:~$ bash cloudy
Cloudy 0.52 Testing
Directory or file to scan:/bin                    
Output path (leave empty for stdout only):/home/example1/output.log
Detection ratio:0/53 File:/bin/bash
Detection ratio:0/55 File:/bin/bunzip2
Detection ratio:0/56 File:/bin/busybox
Detection ratio:0/55 File:/bin/bzcat
Detection ratio:0/55 File:/bin/bzdiff
Detection ratio:0/56 File:/bin/bzexe
Detection ratio:0/55 File:/bin/bzgrep
Detection ratio:0/55 File:/bin/bzip2
Detection ratio:0/57 File:/bin/bzip2recover
Detection ratio:0/57 File:/bin/bzmore
Detection ratio:0/56 File:/bin/cat
Detection ratio:0/53 File:/bin/chacl
Detection ratio:0/56 File:/bin/chgrp
Detection ratio:0/56 File:/bin/chmod
Detection ratio:0/56 File:/bin/chown
Detection ratio:0/56 File:/bin/chvt
Detection ratio:0/57 File:/bin/cp
Detection ratio:0/56 File:/bin/cpio
Detection ratio:0/55 File:/bin/dash
Detection ratio:0/56 File:/bin/date
Detection ratio:0/56 File:/bin/dbus-cleanup-sockets
Detection ratio:0/57 File:/bin/dbus-daemon
Detection ratio:0/57 File:/bin/dbus-uuidgen
Detection ratio:0/56 File:/bin/dd
Detection ratio:0/56 File:/bin/df
Detection ratio:0/56 File:/bin/dir
Detection ratio:0/56 File:/bin/dmesg
Detection ratio:0/57 File:/bin/dumpkeys
Detection ratio:0/56 File:/bin/echo
Detection ratio:0/56 File:/bin/ed
Detection ratio:0/57 File:/bin/efibootmgr
Detection ratio:0/57 File:/bin/egrep
Detection ratio:0/56 File:/bin/false
Detection ratio:0/56 File:/bin/fgconsole
Detection ratio:0/57 File:/bin/fgrep
Detection ratio:0/57 File:/bin/findmnt
Detection ratio:0/56 File:/bin/fuser
Detection ratio:0/56 File:/bin/fusermount
Detection ratio:0/55 File:/bin/getfacl
Detection ratio:0/56 File:/bin/grep
Detection ratio:0/56 File:/bin/gunzip
Detection ratio:0/57 File:/bin/gzexe
Detection ratio:0/57 File:/bin/gzip
Detection ratio:0/57 File:/bin/hostname
Detection ratio:0/57 File:/bin/ip
Detection ratio:0/56 File:/bin/kbd_mode
Detection ratio:0/57 File:/bin/kill
Detection ratio:0/56 File:/bin/kmod
Detection ratio:0/57 File:/bin/less
Detection ratio:0/56 File:/bin/lessecho
Detection ratio:0/57 File:/bin/lesskey
Detection ratio:0/57 File:/bin/lesspipe
Detection ratio:0/56 File:/bin/ln
Detection ratio:0/57 File:/bin/loadkeys
Detection ratio:0/57 File:/bin/login
Detection ratio:0/57 File:/bin/loginctl
Detection ratio:0/57 File:/bin/lowntfs-3g
Detection ratio:0/57 File:/bin/ls
Detection ratio:0/57 File:/bin/lsblk
Detection ratio:0/56 File:/bin/mkdir
Detection ratio:0/56 File:/bin/mknod
Detection ratio:0/56 File:/bin/mktemp
Detection ratio:0/57 File:/bin/more
Detection ratio:0/55 File:/bin/mount
Detection ratio:0/57 File:/bin/mountpoint
Detection ratio:0/56 File:/bin/mt-gnu
Detection ratio:0/56 File:/bin/mv
Detection ratio:0/57 File:/bin/nano
Detection ratio:0/57 File:/bin/nc.openbsd
Detection ratio:0/57 File:/bin/netstat
Detection ratio:0/56 File:/bin/ntfs-3g
Detection ratio:0/56 File:/bin/ntfs-3g.probe
Detection ratio:0/57 File:/bin/ntfs-3g.secaudit
Detection ratio:0/57 File:/bin/ntfs-3g.usermap
Detection ratio:0/57 File:/bin/ntfscat
Detection ratio:0/56 File:/bin/ntfsck
Detection ratio:0/57 File:/bin/ntfscluster
Detection ratio:0/56 File:/bin/ntfscmp
Detection ratio:0/54 File:/bin/ntfsdump_logfile
Detection ratio:0/56 File:/bin/ntfsfix
Detection ratio:0/57 File:/bin/ntfsinfo
Detection ratio:0/57 File:/bin/ntfsls
Detection ratio:0/57 File:/bin/ntfsmftalloc
Detection ratio:0/54 File:/bin/ntfsmove
Detection ratio:0/57 File:/bin/ntfstruncate
Detection ratio:0/56 File:/bin/ntfswipe
Detection ratio:0/57 File:/bin/openvt
Detection ratio:0/57 File:/bin/ping
Detection ratio:0/56 File:/bin/ping6
Detection ratio:0/57 File:/bin/plymouth
Detection ratio:0/57 File:/bin/plymouth-upstart-bridge
Detection ratio:0/56 File:/bin/ps
Detection ratio:0/57 File:/bin/pwd
Detection ratio:0/56 File:/bin/readlink
Detection ratio:0/57 File:/bin/red
Detection ratio:0/56 File:/bin/rm
Detection ratio:0/56 File:/bin/rmdir
Detection ratio:0/57 File:/bin/running-in-container
Detection ratio:0/57 File:/bin/run-parts
Detection ratio:0/56 File:/bin/sed
Detection ratio:0/54 File:/bin/setfacl
Detection ratio:0/57 File:/bin/setfont
Detection ratio:0/54 File:/bin/setupcon
Detection ratio:0/56 File:/bin/sleep
Detection ratio:0/57 File:/bin/ss
Detection ratio:0/56 File:/bin/stty
Detection ratio:0/56 File:/bin/su
Detection ratio:0/56 File:/bin/sync
Detection ratio:0/57 File:/bin/tailf
Detection ratio:0/57 File:/bin/tar
Detection ratio:0/57 File:/bin/tempfile
Detection ratio:0/57 File:/bin/touch
Detection ratio:0/56 File:/bin/true
Detection ratio:0/57 File:/bin/udevadm
Detection ratio:0/57 File:/bin/ulockmgr_server
Detection ratio:0/57 File:/bin/umount
Detection ratio:0/56 File:/bin/uname
Detection ratio:0/56 File:/bin/uncompress
Detection ratio:0/57 File:/bin/unicode_start
Detection ratio:0/56 File:/bin/vdir
Detection ratio:0/54 File:/bin/vmmouse_detect
Detection ratio:0/56 File:/bin/which
Detection ratio:0/57 File:/bin/whiptail
Detection ratio:0/56 File:/bin/zcat
Detection ratio:0/55 File:/bin/zcmp
Detection ratio:0/56 File:/bin/zdiff
Detection ratio:0/57 File:/bin/zegrep
Detection ratio:0/57 File:/bin/zfgrep
Detection ratio:0/56 File:/bin/zforce
Detection ratio:0/56 File:/bin/zgrep
Detection ratio:0/57 File:/bin/zless
Detection ratio:0/56 File:/bin/zmore
Detection ratio:0/57 File:/bin/znew
 
Install Cloudy 0.52 Testing:
1. Download the text file from:

2. Copy the code from between the quote boxes into your text-editor, and save as "cloudy". It doesn't matter where you save it. I saved mine in my user's home folder (eg: /home/example1/cloudy).
3. Install the following dependencies: Coreutils, Sed, Lynx, Find, and Grep. The manner in which you do this will vary from distro to distro, and on most distros at least some of these will be pre-installed (eg: Ubuntu). On Ubuntu and Ubuntu derivatives these can all be installed by typing In terminal:
 
sudo apt-get install coreutils sed lynx findutils grep
 
Run Cloudy:
1. In terminal type:
bash /directory/cloudy
(Substitute "/directory/" for the path to where the "Cloudy" file is saved.)
 
 

Not all files are listed on VirusTotal.com, obviously. In that case you'll get an output saying the file couldn't be found in the database. If the file is found you'll get something like # / #. The first number represents the number of virus scanners that detected the file as dangerous, the second number represents the number of scanners that the file has been scanned with. Ideally you want to see 0 / #.



As before, I was going to submit a copy of Cloudy (the actual script, not the text file) to VirusTotal.com, but it's being slow right now. I'll try again later. :(

Edited by Al1000, 03 September 2016 - 12:26 AM.
delete download link


#5 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 09 November 2015 - 09:19 PM

I've uploaded the latests Cloudy to VirusTotal (I'm in the process of attempting to re-upload), you can view the page here. Alternatively, scan Cloudy with itself. :)



#6 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 13 November 2015 - 02:17 AM

This is a testing release, so there could be bugs, and as such, I'd suggest caution if using the removal feature! So far it's working good for me though. I've been building this on Ubuntu 14.04 Desktop AMD64 but it should work on any distro provided Bash is the shell you're using, and you install the dependencies. Please note that the about.txt file in the archive doesn't list "lynx" as a dependecy, but lynx is.

Cloudy is an open-source anti-malware (in the form of a set of Bash scripts) for GNU/Linux. It can perform cloud-scans, offline-scans, removal of files based on their reported definition ratio, and definition creation. If performing a cloud-scan, the requested files are hashed, the checksums are searched for on VirusTotal.com, the definition ratios listed on VirusTotal.com are then reported. If performing an offline-scan the same thing is done, except the search is performed using a locally stored definition database file. The definition database that comes with Cloudy only includes definitions for some of Cloudy's files, for anything beyond that you must either download a third-party definition file (currently none exist!), or use the definition creator to create a definition file of your own (which would obviously be of limited use to you). Outputs from offline-scans, and cloud-scans can be used with the removal feature to permenantly delete files based on the definition ratio reported for the files during a scan.

Demo Output Of Cloudy 0.60 Testing Doing A Cloud-scan:
example1@example:~$ bash /home/example1/cloudy/cloudy-manager
Cloudy 0.60 Testing
Where is cloudy located (excluding filename):/home/example1/cloudy
Choose a task to perform (cloud-scan, scan, removal, define, quit):cloud-scan
Directory or file to scan:/home/example1/cloudy
Output path (leave empty for stdout only):/home/example1/output.log
Detection ratio:0/52 File:/home/example1/cloudy/About.txt
Detection ratio:0/53 File:/home/example1/cloudy/cloudy-cloud-scanner
Detection ratio:0/53 File:/home/example1/cloudy/cloudy-definition-creator
Detection ratio:0/53 File:/home/example1/cloudy/cloudy-manager
Detection ratio:0/51 File:/home/example1/cloudy/cloudy-offline-scanner
Detection ratio:0/38 File:/home/example1/cloudy/cloudy-remover
Detection ratio:0/52 File:/home/example1/cloudy/Comprehensible Open License (Release 1.0).txt
Detection ratio:0/53 File:/home/example1/cloudy/definitions
Demo Output Of Cloudy 0.60 Testing Doing An Offline-scan:

example1@example:~$ bash /home/example1/cloudy/cloudy-manager
Cloudy 0.60 Testing
Where is cloudy located (excluding filename):/home/example1/cloudy
Choose a task to perform (cloud-scan, scan, removal, define, quit):scan
Definition File (leave empty for default):
Directory or file to scan:/home/example1/cloudy
Output path (leave empty for stdout only):/home/example1/output.log
Detection Ratio:0/1 File:/home/example1/cloudy/About.txt
Detection Ratio:0/1 File:/home/example1/cloudy/cloudy-cloud-scanner
Detection Ratio:0/1 File:/home/example1/cloudy/cloudy-definition-creator
Detection Ratio:0/1 File:/home/example1/cloudy/cloudy-manager
Detection Ratio:0/1 File:/home/example1/cloudy/cloudy-offline-scanner
Detection Ratio:0/1 File:/home/example1/cloudy/cloudy-remover
Detection Ratio:0/1 File:/home/example1/cloudy/Comprehensible Open License (Release 1.0).txt
Definition Ratio:0/0 None. File not in database. File:/home/example1/cloudy/definitions
Install Cloudy 0.60 Testing:

, and extract it where-ever you want it (eg: your user's home folder).
2. Install these dependencies: findutils (find), lynx, sed, grep, and coreutils (sha512sum,sha256sum,seq,sort,paste,tr,tee,wc,cat,rm). The method of installation will vary depending on distribution. On Ubuntu you can install these dependencies using:
sudo apt-get install findutils sed grep coreutils lynx
Launch Cloudy 0.60 Testing:
1. In your terminal type:
 
bash /directory/cloudy-manager
Substitute "/directory/" for the path to where you saved your extracted Cloudy files.

Edited by Al1000, 03 September 2016 - 12:26 AM.
delete download link


#7 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 15 November 2015 - 07:27 PM

I've been doing more testing so I can make a stable release. Cloudy is working great, except for 1 bug. I've also added a new feature, the ability to create a new definition file from multiple existing ones. I figure this could come in handy if a user wants to combine their definition file with one from a trusted associate.
 
I'm not posting the next release yet as that 1 bug needs fixing. The bug is that when using the removal feature with "0" as the max ratio, it removes files with 0+ ratios, which is really weird, and definitely not what you want to happen. If you are using Cloudy 0.60 Testing, don't use the removal feature! I've disabled the download link for Cloudy 0.60 Testing, and won't be re-enabling it since it's affected by the bug.
 
Terminal Output Of The Bug In Action:
Max detection ratio (eg:3):0
Detection Ratio:0/1 File:/home/example1/Downloads/7-Zip 9.20 X86-64 (XP-7).msi Action:Removed
Detection Ratio:0/1 File:/home/example1/Downloads/7-ZipPortable 9.20 Rev 3 X86-32 [PAF](XP-8.1).exe Action:Removed
Detection Ratio:1/1 File:/home/example1/Downloads/BASSFLAC 2.4.2 (BASS)(Windows XP-8.1).zip Action:Removed
Detection Ratio:3/6 File:/home/example1/Downloads/EasyBCD 2.3.exe Action:Removed
EDIT: I belive I've fixed the bug. I think what happened was the numbers in the filenames threw off the detection of the ratio. I've expanded the ratio search to be more specific, and just had a successful test run with the new code. However more testing needs to be done with the removal feature.

Edited by hollowface, 15 November 2015 - 07:46 PM.


#8 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 19 November 2015 - 02:07 AM

For those new to the topic, a quick summary would be, for my own amusement, I started to make an antimalware. Upon realizing it might actually be useful to me, I've continued to work on it, a bit, when I felt like it, and I've been posting the releases online for anyone else whom finds it useful or interesting. Since this isn't something I've put much time into writing, let alone testing, I'd suggest caution when using the removal feature, despite it and everything else seeming to be working well in my current test runs, because if it deletes something, it's gone.

Cloudy 1.0 Stable:
Cloudy is an open-source anti-malware (bash script) for GNU/Linux.

Features:
- Cloud-scan files using VirusTotal.com for definitions.
- Scan files using a local definition database (You'll need a third-party definition database!)
- Create definitions
- Merge definition databases
- Remove files based on scan or cloud-scan result outputs

Install:
1. Download:

2. Extract the ZIP to your desired location (eg: a folder named "cloudy" in your user's home folder).
3. Install the following dependencies: findutils (find), sed, grep, lynx, and coreutils (sha512sum,sha256sum,seq,sort,paste,tr,tee,wc,cat,rm). The manner in which you install these will vary from distro to distro. I've done all my testing on Ubuntu 14.04 Desktop X86-64. On Ubuntu you can install them by typing the following in your terminal:
sudo apt-get install findutils sed grep lynx coreutils
Launch:
1. In your terminal type:

bash /directory/cloudy-manager
(Substitute "/directory/" for the path to where you extracted Cloudy.)
 
For more information read "About.txt" included in the ZIP archive.

Edited by Al1000, 03 September 2016 - 12:27 AM.
delete download link


#9 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 23 November 2015 - 01:09 AM

Cloudy supports doing offline scans using a locally stored definition file, but the definition file that comes with it only includes a definitions for some of Cloudy's files so users whom want to do offline scans need a 3rd party definition file, or to create their own. Creating one's own could be useful to some system admins, but not to most home users. I've started making a definition file for myself (it's not part of the Cloudy project), and figure I might as well upload and share it. It's not particularily useful given it only has 181,865 file definitions currently, but it's still better than the one that comes with Cloudy.

To save on my upload bandwidth, and your download bandwidth I've archived the definition file in an XZ archive, so you'll need to decompress it before you can use it. The uncompressed size is 27.1MB the compressed size is 12MB.

Download Link:

Edited by Al1000, 03 September 2016 - 12:27 AM.
delete download link


#10 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 24 November 2015 - 01:59 AM

Here's an updated copy of my personal definition database file; it now has 240,008 file definitions. Once again I compressed it inside an XZ archive, so you'll need to extract it before you can use it. Hope to add more file definitions to it soon.
 
Download Link:

Edited by Al1000, 03 September 2016 - 12:28 AM.
delete download link


#11 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 24 November 2015 - 07:19 PM

Updated copy of my definition database file. Now has 268,851 file definitions, and is published under the same open license as Cloudy is. It, and a copy of the license are packed into a tar.xz archive to save on my upload bandwidth and your download bandwidth. Hoping to add more definitions soon :).

Download Link:


If you ever want to check how many definitions a definition file has, you can use:
cat /directory/file | wc -l
(Substitute "/directory/file" for the path to and filename of the definition file.)

Edited by Al1000, 03 September 2016 - 12:28 AM.
delete download link


#12 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 26 November 2015 - 11:44 PM

Here's an updated copy of my definition file. It now contains 314,929 file definitions. It, and a copy of the license, are packed into a tar.xz archive to save on my upload bandwidth and your download bandwidth. Hoping to add more definitions soon :). The 500,000 mark is just getting visible in the distance, but I have to squint a bit.

Download Link:

Edited by Al1000, 03 September 2016 - 12:28 AM.
delete download link


#13 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 30 November 2015 - 12:46 AM

Updated copy of my definition file. Now has 362,101 file definitions.

Download Link:

Edited by Al1000, 03 September 2016 - 12:29 AM.
delete download link


#14 Guest_hollowface_*

Guest_hollowface_*

  • Guests
  • OFFLINE
  •  

Posted 04 December 2015 - 02:04 AM

IywkhQz.png
 
Cloudy 1.4 Testing:

The definition creator now uses the same duplicate-checking code as the definition combiner to ensure that even if you scan multiple identical files there is only one entry in the definition file. The removal feature now supports logging to a file, instead of just outputting to stdout. The location of Cloudy is now checked, and quits if the check fails. The check doesn't check for all of Cloudy's components. It checks for cloudy-manager, but no checks are done on the file itself to determine it's the correct file. It only checks to see if the file exists in the user specified location. The dependency list in the "About.txt" file has been updated, it was missing a couple of dependencies. I would still advise caution using the removal feature, as it has had limited testing.

Download Link:


My latest definition file now has 374,893 file definitions. Will add some more soon.

Download Link:

Edited by Al1000, 03 September 2016 - 12:29 AM.
delete download links


#15 Guest_GNULINUX_*

Guest_GNULINUX_*

  • Guests
  • OFFLINE
  •  

Posted 04 December 2015 - 02:19 PM

Very, very nice project!  :thumbup2:






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users