IT Security Assessment Bids
The Good, The Bad and the Ugly
Tips, tricks and shortcuts for evaluating vendors of independent IT security assessment.
This information will help you better understand the bid in front of you and some possible outcomes as you work to shorten the stack of vendors responses. While the information is not intended to be all-encompassing it should help with non-technical considerations. In no particular order, consider the information provided as a signpost at the crossroads of vendor choice.
Hypothesis for third party validation IT security: IT security is a specialty. Normal IT training as good as it is, simply cannot deep dive into all areas of IT Security. Consider the IT hacker, “bad guys” make their living or fame by capturing customer data or corrupting networks. They have nothing else on their plate. They are motivated and rife with time and patience and in some cases, backed by endless resources. With so much at stake, an IT assessment should not be a clash between your IT department and the outsourced assessment vendor. By analogy, a primary care physician refers her cancer patient to an oncologist. Your IT specialist should be able to help you with a complicated environment, cooperate with them and use their findings as a tool for remediation and staff training. If your vendor is punitive in their approach to assessment, then you have the wrong vendor.
- Expensive, economical or cheap –Consider cost weighed against value rather than perception of vendor brand. Price has never been a good indicator of quality; we’ve all paid for something that did not live up to expectation. Without value it really does not matter what you spend. Vendor selection should meet some basic conditions, a few of which are: 1) complimentary business style, 2) aligned interests, 3) actionable and valuable deliverables. You’ll want to define the vendor’s meaning of “reasonable” costs? Does the bid itemize services or does it use time and materials as a basis for costs? Which of these options would your institution prefer? Are there unnamed “out-of-pocket” expenses? Is there a sliding fee structure based upon the personnel skills assigned to your engagement? The short of it: Review enough bids to compare scope of work, methodology and references. Peer references often provide the objective assurance of value needed to make an informed decision. Regulatory or internal targets met, combined with a reasonable fee for better than average work equals the best fit for most organizations.
- Big shop, all in one shop or specialty shop?What you want is just as important to know as what you do not want. Take a look at bids from both large and small firms. Do you want to work with a vendor in which your assessment is one of many that will be project-managed by an army of souls committed (or not so committed) to the corporate mission? Request proposals that match your institution’s requirements, work style and budget. If your engagement will be handled by a network admin who just got done installing workstations last week and will complete your security assessment this week, this would be a condition of trying to be all things to all people. Will you accept outsourced personnel, an intern, and a certified security engineer? If your vendor has a reputation of low turn-over, a seasoned veteran staff, does not try to be everything to everyone, is big enough to get the job done, but small enough to care then you likely have a good option in front of you. Avoid firms that remediate their own assessment findings, regardless of separation of duties. Examiners guidance might let this pass as an oversight or it can come down hard without any notice. Avoid the fox in the hen house.
- CPA Firm vs. IT Security boutique firm –Accountants are expertly trained to handle all things financial and principally this work is above reproach. When an assessment firm or CPA firm also offers IT services or resells other non-related services the potential for conflict of interest exists. Exposing your business to the level of detail that is common of a financial audit and then allowing the same group full access to the IT environment could be giving away too much control to one vendor. Specialists in Information Security are equipped to handle their craft in spite of the rapid change within their bailiwick. Boutique shops allow you access to the top levels of the organization along with certified specialists. The take away: finance specialists should manage accounting; IT security professionals should manage IT assessment.
- Certifications –What are the certifications of the engineers you’ll hire and how valid are they for the work being contracted? Make sure that the potential firm’s knowledge bucket is filled with the diverse “smarts” relevant to your industry. Experience, leadership, collaboration and effectiveness are bellwethers that echo through many solid business relationships. Since engineers and auditors spend time with you and your team, you’ll want professionals who are objective, affable and able to communicate at your level. In addition, relevant certification is an integral component of an expertly executed engagement.
- Hidden Agendas –Beware of a really inexpensive assessment. Some of these will include either a sticky contract or hidden agreements that combine a host of hardware and services that appear following the assessment. Carefully read “boilerplate” language often relegated to the back pages of the bid, request clarification of any items that are not crystal clear. If the price seems out of whack, either too high or too low the reason could be that your organization is making future commitments to services that were never part of the intended scope. If the bid makes it to the short list, be sure to contact past customer references, ask the vendor for sample reports, review corporate validation, and find public relations notices to carefully vet the bid.
- Multi-year contracts –what are the benefits of long term commitments? Read the escape clause in case the work is not what you intended. Ownership changes (yours or the vendor’s) over the course of a longer term contract may require legal action to ultimately release you or the vendor from future commitment. Your vendor should clearly spell out consequences, if any, for early termination. Provided the selected vendor works well the first year, then consider a multi-term proposal if bank policy permits. The benefit is a flattened cost of assessment year-over-year. This also lessens the burden of vetting new vendors each year. Plus, you’ll be able to show positive trending using the same vendor as your benchmark.
- The go away bid –Why would a vendor delay a bid? What does this indicate about their work? Drop this one from the mix and you have likely just dodged a bullet. Just because a bid takes weeks to deliver does not make the information better. Realistic timeline: Initial phone call or email with interest in receiving a bid should only take a few minutes. Unusual or special requirement may add 30-minutes for a Discovery Call. Once the high level information is exchanged a bid should take no more than a week, at the most to arrive in your email. Seek vendors who are eager to supply you with their most accurate statement of work in a reasonable timeframe. If time is not money in your world, then wait as long as you like. Otherwise, consider only those bids that are well constructed, comprehensive, understood by all levels of technical ability and delivered in a timely fashion.
- No bid at all –Why would a vendor delay a bid? What does this indicate about their work? Drop this one from the mix and you have likely just dodged a bullet. Just because a bid takes weeks to deliver does not make the information better. Realistic timeline: Initial phone call or email with interest in receiving a bid should only take a few minutes. Unusual or special requirement may add 30-minutes for a Discovery Call. Once the high level information is exchanged a bid should take no more than a week, at the most to arrive in your email. Seek vendors who are eager to supply you with their most accurate statement of work in a reasonable timeframe. If time is not money in your world, then wait as long as you like. Otherwise, consider only those bids that are well constructed, comprehensive, understood by all levels of technical ability and delivered in a timely fashion.
- All the money upfront – Really?Your ears should be perking up along with the hair on the back of your neck when you see this. Some companies require payment in full before any work begins. Maybe they offer a discount for this policy, does this make you feel warm and fuzzy about the cash flow of an organization with whom you will entrust your institution’s information security assessment? What recourse have you got if they fold up the card table and skip town? Why not donate to charity; at least you’ll get a tax write-off.
- Contractor work –Who does the work once the contract is signed? Is there a team of in-house professionals working in peer review mode to complete the engagement based on specialized and deep skillsets or is your assessment in the hands of a third-party contractor with absolutely no accountability to you? What happens when you have questions, concerns or follow up later on? Wouldn’t it be best handled directly by your interested party and preferably by the engineer who performed the engagement?
- Compare Apples to Apples –If there are oranges in the basket, are they part of the requested menu? When looking at IT security assessment bids, there are occasions when the scope of work is worded inconsistently. Vendors should be happy to receive requests for clarification of their scope. This is an opportunity to assure you that your requirements are equal in all ways as you compare vendor solutions. If there are differences in the scope of work to be performed be certain that your requirements are met first. If you end up with “extras” in the proposal and the pricing, process, and results would be in your favor, the vendor should make the short list of contenders.
- A rose by any other name –technical jargon like marketing jargon can be enigmatic If your contract is full of flowery IT language that really doesn’t account for the work being completed it may be more trouble than it’s worth to read. No matter what “it” is called, the goals, scope and methodology should include clearly stated, concise language that can be understood by everyone.
- Vendor Due Diligence –Third party vendor management lays out the requirements to properly assess vendors. These requirements are an assurance that your vendor will be around to complete the work and follow up with you should the need arise. Your regulatory body will have concise requirements for the specific service provider you seek. Be sure to follow either your organization’s internal policies or your regulatory guidance when selecting vendors.
- When should a new vendor be contracted?The best time to move to a new vendor can be a complicated decision. Sometimes your regulators may “suggest” it. Your board of directors may influence the change, or internal policy can require that a vendor be used for a specified period of time. However, in the absence of guidance here are some of the indicators that may lead to change. 1) Year-over-year findings are fewer and ratings are flat. 2) The vendor organization has changed hands or experienced high staff turnover. 3) More obvious telltales may include unfavorable news stories or pending legal action. Barring the unsavory, vendors that remain in good standing can always be included for special projects, or incorporated back into the mix in the future.
- How many bids should you be reviewing?Bids are or should be no cost and no-pressure. Go ahead and accept bids from as wide a cross-section of vendors as time permits. This will allow you to familiarize yourself with the services and measure them based on your selection system. Consider your schedule and bandwidth as the measure for the number of bids to accept. Communicate (email is fine) bid review status in a timely fashion to participating vendors, whether they made the short list or not. Their response should always be gracious and express appreciation for the opportunity to compete. Anything less is a reason to remove them from future consideration until something changes.
- Engagement Timing –If a vendor books out at least 90-days in the future, that’s actually a good sign. It means that your peers are selecting that vendor too and likely because they are competitive and competent. Consider your schedule when requesting bids. If you’ve been tasked with receiving vendor bids, taking a first pass to determine which ones make the short list and if the time to engagement is short, there may be times when a vendor cannot schedule in short order. While most firms want to do their best to work with your schedule, longer term planning will assure that your schedule can be managed to your expectation rather than having to accept the number two or three choice for vendor.
- Do you want the truth?Jack Nicholson in the movie: “A Few Good Men” roared “You can’t handle the Truth!” depicting a war weary marine officer during a court martial. Luckily, the less dramatic truth is that your IT assessment results are based on a point in time, they are objective and without malice. If year after year the board has allocated proper budget and acknowledged that work should be completed to mitigate findings then IT staff should be able to demonstrate their course of action. New risks and vulnerabilities present almost daily. The landscape from year-to-year can be peppered with findings that did not exist previously. If your assessment is nothing more than a rubber stamp on a check list there is no value. Choose a vendor with conviction, not one that will let things slide because it could be painful to hear the truth. Given the stakes, truth in the light of day will always be a better outcome.
Authored By: Bill Brock