March 2005
Selecting Application Security Vendors
by Jose Varghese, CISSP, BS7799 LA | 
Discuss this article »» (2)
Traditional security has always been focused on perimeter defense. With most of the organizations having strengthened their perimeters with Firewall, VPN and intrusion detection systems, attackers have shifted their focus to the application layer. Most of these attacks are far more damaging that network layer attacks and primarily focus on the weaknesses in the application like poor input validation; insecure sessions management etc. For effective security, it is important for the enterprise to ensure that all business applications are tested for security as rigorously as they are tested for functionality and performance before they are deployed in production
There are multiple aspects to be considered while selecting the application security vendor: technical skills, delivery method and customer references.
Evaluating Technical Skills
Security domain knowledge: Application security is an evolving field with rapid advances both in attack and defense techniques. For best results, it is important we select a vendor who is abreast with all the developments in this area. Although there are no official certifications that demonstrate experience in application security testing, there are several global forums where advances in application security are actively shared and debated. Some of these forums include OWASP, Web Application Security consortium etc. It would be appropriate to checkout if the vendor does active research in this area and is an active participant in global security forums. This will be a demonstration of expertise as well as commitment to continued excellence in this field.
Software development experience: The testing team should have sufficient experience in software development and be familiar with the SDLC cycle. This is required for quicker understanding of the software code, if we are doing a white box testing, (See Box) and also for providing implementable solutions for identified weaknesses.
Business domain expertise: The test cases for application security testing are mostly developed based on the business logic implemented by the software. Hence it is important that the testing team understands our business. Check if the project team members have experience in testing similar type of applications for other clients. This would reduce time and effort for understanding the applications. This will also ensure that their testing approach and methodologies have been fine tuned to consider all risks in our business area.
Evaluating Delivery Methodology
Testing methodology: There are different approaches to testing (as shown in Box). Look out for the comprehensiveness and efficiency in the testing approach. Some vendors use a combination of black box and white box testing for achieving the best results. Similarly there could be a mix of automated tools and manual verification. The exact method of testing will depend on the nature of the applications and time available for testing. An experienced vendor understands the uniqueness of each application and will suggest a customized approach based on the requirement.
Reporting capabilities: Ensure that the reports contain all information regarding the vulnerabilities discovered and possible solutions. It would be good to take a look at sample reports. Does the vendor categorize the vulnerabilities based on threat and risk profiles? Are detailed steps to fix the vulnerabilities part of the report?
Solution support: Check if the vendor provides technical support for implementing the corrections that have been recommended in the report. In some cases it may be required to do a re-verification after implementing the fixes. This will ensure that the vulnerability has been removed and also verify that these changes in turn have not created new vulnerabilities.
Application security testing requires high level of technical skills and a fair understanding of business process. Conduct reference checks to validate the claims made in the proposal.
If feasible, start small. Provide single applications for testing. This will give us an opportunity to evaluate the effectiveness of the vendor before investing money and effort in large assignments.
Various Approaches to Application Testing
- Go through the entire code and see if there are any instances of insecure design or coding practices (white box).
- Test the application by using sample login credentials (grey box).
- Try standard attacks without having access to code or having any valid user credentials.(black box).
Discussion is open for this article —
there are 2 reader comments.
Add yours.