Leveraging Open Source for Continuous Application Security at Agile Continuous Integration and Continuous Delivery environments
This post is an abstract of my submission to nullcon 2015.
With more and more web applications enabling us to converse, communicate, conspire, collaborate, contribute, capture, compute, credit, conjure, and even keeping us content; more and more trivially insecure vulnerabilities are left to be exploited. With development teams always confronted with deliverables and ever faster imminent deadlines, they always see penetration testers as blockers because when a developed application is penetrated and vulnerabilities are identified the effort to fix them surges and it just would have been downright child’s play, had those bugs been identified early in development cycle when features of the application were being built and tested. As deployment dates near, most raised security bugs still remain raised, the critical and some lucky high risk bugs are fixed, but stakeholders accept medium and low risk bugs as technical debt, our vulnerable application meets the world wild web.
The early feedback cycles introduced by Continuous Integration systems have proven that when software development teams are faced with bugs, and build failures, they have always stepped up with early fixes. In the same way a system with Continuous Security aims at a transparent security governance of application development, making security visible at all levels – development, quality, compliance. As ‘tools in the market can’t oust human intelligence at penetration tests’ stands as an immutable fact ‘tools are used to automate and identify recurring and routine security bugs and complement manual audits’ also stands true. Sure, there are security wary development firms that use penetration testing tools, part of their development cycles. However, there hasn’t been a rapid adoption of automated tools in the rapid and agile application development processes. The primary answer could be cost, time, effort, lack of security awareness, fear of introducing more bugs trying to fix an existing bug, habit of responding to latest threats instead of proactive measures. Even some of the free and open source offerings lack support, active development community, documentation, usability, scripting support.
Bugs are something that frightens every developer and tester. Working with development teams of various sizes it is evident that the earlier the bugs are identified the easier they are to fix. While the proposed idea might look familiar or said before or may be even some of the organizations have expensive tools deployed to identify bugs every night, they are no extra steps taken to get a developer’s attention. Nothing could be more embarrassing for a developer to have broken a build and notified about the bugs. Getting a developer’s attention with build notifications and failures, getting the identified bugs fixed, keeping the source control free of the common security vulnerabilities at the cost of free and open source software with a generic platform agnostic approach is different and innovative about this paper. Also the sample source that accompanies this abstract would help many needy security analysts and developers to understand how Continuous Application Security could be done, and actually implement Continuous Application Security for their web applications.
This paper proposes a generic framework/approach with open source options to integrate vulnerability analysis as a daily chore during web application development and maintenance. The fundamental idea is to integrate a custom pipeline build script or use the boilerplate code accompanying this paper into the build system; the custom pipeline build script henceforth referred to as the CPBS. Based on the vulnerabilities found the CPBS would update the build system for success or failure. Conditional action should be taken to keep the new checked in code into the version control system or discard it or to stop a production deployment for continuous integration or continuous deployment setup respectively. The CPBS sequence follows a work-flow of starting our security tools on a port to listen; attempts a health check on the website Url and keep polling until the website services are up; excludes urls that destroy active sessions; if a functional test suite is available, monitors and waits for the functional tests to complete; lets the security tool analyze the traffic and sense vulnerabilities based on passive scanning; starts a webdriver script to create an active session for the application; apart from the urls obtained while the functional tests were running, fires up the spiders to crawl for more resources; sets the desired scan policies and starts the scan; sends success or failure code based on the results.
Embracing automated functional tests within the Vulnerability Analysis process, handling authenticated resources and login protected websites, handling intensive client side scripts with JSON or single page web applications, handling CSRF tokens, handling iterative and incremental scans, false positive management, promoting security culture and governance in the organization, managing identified bugs in tracking systems, keeping the code in version control systems free of security bugs are some of the key areas of focus.
So we are talking about penetration testing and we are talking about vulnerability analysis.
Think about a bank job – movies like Dog Day Afternoon, Inside Man, American Heist or the 1995 movie Heat. Before the bank robbers penetrate into a bank, they recon the place for days and days together, and look for vulnerable spots, study the building schematics so they could make use of a weakness or a couple of weaknesses to penetrate in to the bank – get in and out sometimes without leaving a trace and sometimes with damages to the banks’ property.
Think about a web application that has a weakness in output encoding, an attacker could exploit this weakness and try to hijack sessions, do a Denial of Service, change web page content, include key loggers, steal information, serve malware and so on. Think about an another application that takes user input, does not validate it and concatenates into a SQL statement, an attacker could exploit this weakness and try to access confidential data from database, hack into the company’s corporate network, or sabotage systems.
Identifying vulnerabilities like encoding mistakes (XSS), concatenation mistakes (Injection) are done during a vulnerability analysis. Identified vulnerabilities could then be exploited leading to a successful penetration.
Often in terms of computer hacking, penetration testing (aka pen testing) is an activity where a person (that is called a penetration tester) tries to penetrate (or hack into) a particular resource/system. In order to do that the penetration tester often analyzes the system/resource for vulnerable spots that could lead a way in. Hence vulnerability analysis (VA) is done to identify weak spots in an application. The results of a vulnerability analysis (VA) could be used for an effective penetration testing (PT).
This is a quick and dirty blog for those that are new to Eclipse IDE and want to try tweaking the OWASP Zed Attack Proxy’s code. I must say that that you might stumble upon this well written guide titled “Building OWASP ZAP Using Eclipse IDE for Java… Pen-Testers” here - http://www.taddong.com/docs/Building_ZAP_with_Eclipse_v3.0.pdf . First time I was trying to build ZAP with Eclipse this guide was my complete reference. However, OWASP ZAP’s code was recently move to GitHub in the month of May-June 2015 rendering that guide obsolete and my OWASP ZAP Eclipse workspace – connected to google code SVN – a little defunct. Raul Siles, the author of the above guide would update it for changes with respect to the GitHub move.
Recently I was trying to download OWASP ZAP’s code from GitHub and build it because the existing code from SVN (google code) went obsolete. I am not an advanced Eclipse user or Java developer and I was a little lost trying to clone the new OWASP ZAP GitHub repo to my Eclipse. As I was trying, I took screenshots and ended up posted in this blog. Remember, this blog is not a step by step instruction, but it is a quick and dirty steps (5 major steps) to get OWASP ZAP’s code running in your Eclipse IDE.
Glimpse through the articles titled
- Building OWASP ZAP Using Eclipse IDE for Java… Pen-Testers
- Building ZAP (https://github.com/zaproxy/zaproxy/wiki/Building),
- Downloading and Building OWASP ZAP source from Github using Eclipse IDE (this article)
and I am sure you’d get ZAP running on your Eclipse IDE.
… from https://eclipse.org/downloads/. If you are confused which edition to download, pick the Eclipse IDE for Java Developers
When you open Eclipse for the first time choose the default workspace and proceed. If you’d like create a workspace such as workspaceowaspzap like I did. Refer to Raul Siles guide for workspace screenshots.
Make sure you have EGit plugin installed. If you are a prime time command liner with Git you may not need this plugin.
If you have downloaded Eclipse from Eclipse for Java Developers, then please ensure in the Eclipse Installation Details you have the below three components highlighted
Eclipse Git Team Provider
Java Implementation of Git
Mylyn Versions Connector: Git
At the time of this writing Eclipse IDE for Java Developers comes with all required plugins to work with Git ( and hence GitHub)
Add a Git Perspective
… to view Git Repositories and stuff..bla bla
Hit the Open Perspective button at the right top corner
Choose Git at the Open Perspective Dialog
Hit OK to view the Git Repositories view.
Tip: From time to time you could hit the Java perspective to view the Java related tools and views, you could hit the Git perspective to view your Git Repositories.
If you look at the workspace that we choose when opening Eclipse, in Windows Explorer now it just has one folder named .metadata. Time to download the code from https://github.com/zaproxy
Downloading the OWASP ZAP’s code
Choose File –> Import
Select Team –> Team Project Set. Hit Next.
In the Team Project Set Dialog, Input the Url –
and hit Finish.
Tip: Always refer to the recent project set Url available at https://github.com/zaproxy/zaproxy/wiki/Building
Wait for the ZAP projects to be downloaded and built
Watch the progress as the Git Repositories view would show projects as and when they are downloaded
Once all the ZAP projects are downloaded, your workspace the Git Repositories view should look like below. The approximate size of the workspace with all the ZAP coded summed up to 2.27 GB for me (on July 4 2015).
Run ZAP’s source and start playing (and contributing)
Switch to the Java perspective
In the Package Explorer, right click zaproxy and choose Run As –> Java Application
Eclipse would search for the Main types. In the Select Java Application dialog choose ZAP and hit OK
Witness the Console Logs
Tip: You can also start ZAP by hitting the play button in Eclipse
If you encounter any problems, try fixing it yourself first – spend a day or two , as a last resort – post at the ZAP Developer group here – https://groups.google.com/forum/#!forum/zaproxy-develop
The OWASP Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. Originally a fork of the Paros Proxy project, ZAP targets a wide range of software professionals right from a software developer to a penetration tester working on any platform that supports Java. Equipped with a myriad a features and support for custom addons, ZAP is fully documented in an easy to understand language.
We would see a demonstration of how to set up and how to use it all.
Starts at Saturday November 22 2014, 12:15 PM. The sessions runs for about 1 hour.
Practical Security Testing For Developers Using OWASP ZAP at Dot Net Bangalore 3rd meet up on Feb 21 2015
|Title||Practical Security Testing for Developers using OWASP ZAP|
|Abstract||Every time an application faces the world wide web, it inherently becomes vulnerable to attacks. The attackers could be script kiddies, joyriders, turning from hobbyists to downright hostile. The earlier in the development cycle you find the vulnerabilities, the better they are to fix and test. OWASP ZAP is a free and open source penetration testing tool for finding vulnerabilities in web applications; widely used by security professionals, it is also ideal for anyone new to web application security and includes features specifically aimed at developers. This session shows/demonstrates some attacks against web applications and how OWASP ZAP could be used to find those vulnerabilities, both manually and by automated builds.|
|Gist||See live attacks against web applications and how OWASP ZAP could be used to find those vulnerabilities, both manually and by automated builds.|
|Time & Venue||21 Feb 2015 @ Dot Net Bangalore 2nd meet up|
Ok, may be the title is a bit misleading. The below post is going to show you a few workarounds that you can use to generate (or mimic) a html report when running OWASP ZAP automated in a headless mode.
Zed Attack Proxy (ZAP), as of version 2.3.1 stable release, provides options to generate an Xml report and an html report for the alerts found. We could get these two reports from the UI via the Report menu.
One of the unique features of ZAP is the REST API that aids in automating scans. When running ZAP in daemon mode and pivoting ZAP from the REST APIs, there is a number of ways to harvest the alerts identified by ZAP.
Here’s a couple of them:
- alerts based on id (http://localhost:7070/UI/core/view/alert/)
- alerts for the baseurl (http://localhost:7070/UI/core/view/alerts/)
- xmlreport (http://localhost:7070/UI/core/other/xmlreport/) [Same as clicking Report –> Generate Xml Report…]
Using the API and your favorite programming language, you could very well gather the alerts and decide what you want to do with it.
One convenience is just pulling a html report out of the scan and mail the application owners or ourselves so they could be fixed. There is an enhancement request open in the projects page – https://code.google.com/p/zaproxy/issues/detail?id=1355
Until the feature is available in the next stable release, I decided to love with a couple of workarounds. There is a workaround in python, there is also a workaround in an exe format (written in C#). They are not perfect in terms of formatting, but they are close enough to the original html report generated through the ZAP UI.
Download from here – https://github.com/gmaran23/HtmlReportThroughZapAPIs
A python script to inject an xslt stylesheet into the xml report file generated through the API. Use the sample function to insert the sample (ZapReport.xslt) into the xml file generated. Needless to say that the ZapReport.xslt and the xmlreportfile should be in the same location.
If needed to email the xml report, we can zip the xml report with the xslt file, so when unzipped the xslt file resides in the same location as the xml file.
When the xml file is opened in Firefox, or IE –> see the formatting in action! (Does not work in chrome though )
There is also another xslt template available from https://code.google.com/p/zaproxy/source/browse/trunk/src/xml/report.html.xsl
Use the command line program named XmlToHtmlWithXSLT.exe (requires .Net 4.0) as below to obtain a html report as output.
If you don’t have .Net 4.0, use the source below to recompile to any .Net version.
Untill, the enhancement request (https://code.google.com/p/zaproxy/issues/detail?id=1355) is completed, these are some workarounds that I personally could live with, may be you can too, for your automating needs .
Plug-n-Hack introduces and proposes new standards to integrate security tools with the browsers, enabling communication between them. OWASP ZAP has inbuilt support for Plug-n-Hack (pnh) which allows you to configure Firefox to change it’s proxy settings so that OWASP ZAP could watch the FireFox traffic.
Configuration is a child’s play. Point your browser to the ZAP proxy address, follow instructions, and you are done. Just like the gif image below.
This changes Firefox to use a proxy configuration provided via the http://localhost:7070/proxy.pac file.
But, out of curiosity if you went and changed the FireFox proxy settings to No Proxy or Auto-detect proxy settings for this network or Use system proxy settings then the FireFox traffic would not be proxied through ZAP which is expected, Right? That works just fine.
However, when you want Firefox traffic to be proxied through ZAP again, you would copy paste the ZAP proxy address (http://localhost:7070/pnh) in Firefox again, and Firefox would then say A provider with this name has already been configured
What is your expectation now?
I don’t know, as a user my expectation when I pasted the http://localhost:7070/pnh url in Firefox is that it should configure my Browser to route it’s traffic via ZAP. But that does not happen.
How to fix?
You can override the proxy settings yourself. Or you could actually use pnh to clear and remove a configuration and then point Firefox to http://localhost:7070/pnh
Shift + F2 in Firefox and then two commands for you:
Windows changes with every version and the UI gets twisted and twisted, for a computer user that works with command line for administrative tasks, things are pretty much the same.
If you want to gather the list of MAC address on a particular computer run wither of the below commands:
2. find on ipconfig –all.
Originally posted on chentiangemalc:
Process Explorer the tool we’ve all come to love as “Task Manager on Steroids” has been for many IT pros one of the essential tools in their troubleshooting toolkit. Process Explorer was originally released in 1998 under the name NTHandlEx. Here is a screenshot of version 1.22. Notice the lack of processes in Windows NT 4.0!
By version 2.01 it had been renamed to HandleEx added some more process properties and kill feature.
It wasn’t until 16 June 2001 when Version 5.0 came out that it got renamed to Process Explorer. ( I was hoping to have a screenshot of this version as well but couldn’t find it anywhere…) In any case as of May 2011 with version 14.12 the tool has come a long way to be one of the most advanced “task manager” tools available:
However an open source project has been working on a competing product since…
View original 1,637 more words