At Strobes, our mission has always been to empower organizations with cutting-edge solutions for Penetration Testing as a Service (PTaaS). While we greatly value third-party evaluations, accuracy is critical to maintaining industry standards and credibility. The recent GigaOm Radar Report for PTaaS highlights Strobes as a player in the market but contains several discrepancies that fail to reflect the full scope of our capabilities.
In the cybersecurity industry, where trust and credibility are paramount, reports and evaluations by analysts hold considerable influence. These publications can shape perceptions, guide purchasing decisions, and even define the trajectory of brands in the market. But what happens when a report gets it wrong? What happens when inaccuracies are published, despite clear evidence to the contrary? Unfortunately, this is not a hypothetical scenario—it’s a reality we at Strobes are grappling with right now.
What exactly happened though?
Recently, GigaOm released its Radar Report for Penetration Testing as a Service (PTaaS), where Strobes was evaluated. While we were initially hopeful about the opportunity to showcase our capabilities, the final report fell woefully short of accurately representing our platform. Here’s what happened:
The GigaOm analyst shared the first draft of the report, riddled with inaccuracies, and despite our immediate response with detailed proof and documentation, they chose to ignore the evidence. To make matters worse, they did not share the final draft or the final output with us before publishing, leaving no opportunity to address lingering issues or correct further misrepresentations. As a result, the final report remains largely unchanged, perpetuating these falsehoods. This disregard for facts has not only undermined our efforts but has caused significant damage to Strobes’ reputation, misrepresenting our capabilities and tarnishing the trust we’ve built with our audience.
To give an example of the inconsistency of the Gigaom radar report –
At first, in the “Strengths” section of Strobes –
Here, they gave a 4 out of 5 for “Flexible Pentesting”
In the same report, they have given 2 out of 5 for “Flexibility”
This inconsistency is not just confusing; it’s outright illogical!
How can a platform simultaneously excel in flexibility for penetration testing, as explicitly stated, yet score abysmally low on the very same parameter in the broader evaluation? Such a blatant contradiction undermines the credibility of the report and raises serious questions about the rigor of the evaluation process. It feels careless and diminishes trust in the objectivity of the analysis.
Evaluations like these need to be consistent, transparent, and aligned with the evidence presented. Instead, this inconsistency not only misrepresents Strobes’ capabilities but also creates confusion for potential customers who rely on these reports to make informed decisions. Such contradictions are not just an oversight, they are damaging and, frankly, unacceptable.
In this document, we aim to set the record straight by addressing the inconsistencies and presenting the true capabilities of Strobes Security that make us a leader in PTaaS.
1. Automated Workflows
What the Report Claims: Strobes received a 2/5 score, with the report stating that ‘workflows are limited to enhancing communication.’
The Reality: Strobes provides extensive automation capabilities designed to simplify and streamline security operations, including:
- Integration-Based Automated Workflows: Seamlessly connect Strobes with 120+ solutions for end-to-end security processes.
- Auto-Routing of Findings: Automatically route vulnerabilities to the appropriate teams using SDLC integrations.
- Automated Vulnerability Management: Reduce manual effort with workflows that automatically manage vulnerability lifecycles.
- CI/CD Pipeline Automation: Embed security testing directly into your CI/CD pipelines for proactive vulnerability management.
2. Integration Ecosystem
The GigaOm Radar report critiques several platforms, including Strobes, for having limited integration capabilities. However, a closer examination of publicly available documentation highlights the extensive integration ecosystem Strobes offers. Key features include:
- 120+ Security Tool Integrations: Comprehensive support for tools across vulnerability management, ticketing, collaboration, CI/CD, and cloud security.
- Native DevSecOps Integrations: Seamlessly embed security into development workflows with tools like GitHub, GitLab, Jenkins, and Azure DevOps.
- Extensive API Support: Enable custom workflows, real-time data exchange, and advanced automation to meet unique organizational needs.
- Bi-Directional Synchronization: Ensure consistent data flow across systems, reducing errors and improving collaboration between teams.
These capabilities clearly demonstrate Strobes’ ability to unify security and development efforts, contradicting the report’s assessment of limited integration functionality.
3. SDLC Integrations
What the Report Claims: While acknowledging Strobes integrates with 120+ tools, the report assigns low scores for SDLC integration capabilities.
The Reality: Strobes offers deep integration capabilities across the SDLC ecosystem, including:
- Ticket Management: Seamless integrations with Jira and ServiceNow.
- Code Scanning: Compatibility with popular tools like GitHub and GitLab.
- CI/CD Pipelines: Full support for tools like Jenkins and Azure DevOps.
- Notifications: Real-time updates via Slack and Microsoft Teams.
Here are the product snapshots –
4. Built-in Scanners
What the Report Claims: The report understates Strobes’ scanning capabilities.
The Reality: Strobes provides a wide range of native scanning tools to address various security needs, including SAST, DAST, SCA, SBOM
- Dynamic Scanners for real-time testing.
- Infrastructure Scanning Tools for network and server vulnerabilities.
- Network Vulnerability Scanners to detect and remediate external threats.
- API Security Testing Tools for robust API protection.
- Container Security Scanners to safeguard cloud-native applications.
Here’s the product screenshot of our native scanners –
5. Reporting Capabilities
What the Report Claims: The report downplays Strobes’ reporting features.
The Reality: Strobes delivers industry-leading reporting capabilities, including:
- Customizable Report Templates to meet organizational and regulatory needs.
- Compliance-Mapped Reporting for standards such as ISO, GDPR, and PCI DSS.
- Executive Dashboards offering high-level insights.
- Detailed Technical Reports for operational teams.
- Automated Report Generation to save time and resources.
6. Enterprise Capabilities
What the Report Claims: The report positions Strobes as suitable primarily for smaller deployments.
The Reality: Strobes is an enterprise-grade solution offering:
- Scalable Architecture to support large-scale operations.
- Enterprise-grade security features to protect sensitive data.
- Multi-tenant support for managing multiple projects seamlessly.
- Role-Based Access Control (RBAC): Ensure only authorized personnel access critical systems.
- Enterprise SSO Support: Simplify user management and authentication.
7. Forward Mover Classification
What the Report Claims: Strobes is classified as a ‘Forward Mover,‘ given its relatively slow rate of development in key areas over the last 6-12 months.
The Reality: Strobes has demonstrated unparalleled agility and innovation, with:
- 1,199+ Releases This Year: An impressive pace of development.
- 120+ Major Feature Releases: Continuous platform improvements.
- Enhanced Automation Capabilities: Regular updates to meet evolving market needs.
- New Integration Additions: Expanding our ecosystem to include the latest tools.
Overall Rating
What the Report Claims: Strobes received an average score of 2.9, which seems inconsistent with its capabilities, especially compared to competitors with similar or fewer features.
The Reality: Strobes provides a feature-rich, enterprise-grade PTaaS solution that goes beyond traditional offerings. When compared to competitors, Strobes excels in areas such as:
- Automation and integration depth.
- Scanning breadth.
- Reporting customization.
- Enterprise scalability.
The Hidden Cost of Inaccuracies
Inaccuracies in such reports don’t just harm a brand’s reputation; they mislead decision-makers who rely on these evaluations to choose the best solutions for their organizations. Here’s the ripple effect:
- Erosion of Trust:
- When reports fail to reflect reality, it undermines trust in both the brand being misrepresented and the analyst firm itself.
- Missed Opportunities:
- Misrepresentation can steer potential customers away, pushing them toward competitors who may lack the same level of innovation or capability.
- Market Confusion:
- Inaccurate reports create confusion in the market, making it harder for organizations to identify the solutions that truly meet their needs.
Impact on Procurement Decisions
The influence of analyst reports, like the GigaOm Radar, on procurement decisions cannot be overstated. These reports often serve as a cornerstone for organizations determining which vendors to engage with, and inaccuracies within them can have far-reaching consequences:
1. RFP Requirements Mirror Analyst Reports
Many organizations structure their Request for Proposals (RFPs) based on the criteria highlighted in analyst reports. Misrepresentations in these reports can lead to RFP requirements that unfairly exclude innovative vendors, like Strobes, who may not score high due to flawed evaluation processes but offer superior capabilities.
2. Vendor Shortlists Are Influenced by Rankings
Procurement teams often create vendor shortlists by relying heavily on analyst rankings. A lower score, such as the one Strobes received for “flexibility,” can unjustly disqualify it from being considered, despite its proven technical excellence and adaptability.
3. Budget Allocations Skewed Toward “Leaders”
Companies frequently allocate their budgets toward products and vendors labeled as “leaders” in these reports. This approach not only undermines emerging innovators but also perpetuates a market bias that rewards marketing presence over genuine technical advancements.
4. Innovation Overlooked in Favor of Market Presence
Analyst reports often prioritize well-established vendors with significant market share, even if their solutions lack the innovation required to address modern challenges. This bias discourages forward-thinking procurement teams from exploring new and potentially superior solutions.
Inaccurate analyst reports like the one published by GigaOm do more than misrepresent; they actively harm innovative companies like Strobes by creating misplaced perceptions and biases in the market. The damage extends beyond sales—it affects trust, credibility, and long-term growth. It’s crucial for Strobes to actively challenge these inaccuracies, communicate its true value to stakeholders, and advocate for fairer evaluation processes to prevent similar issues in the future.
Why Accuracy Matters?
Reports like GigaOm’s are not just evaluations; they are narratives that shape how brands are perceived in the industry. For us, the narrative in this report is not just incorrect—it’s harmful. It dismisses the years of innovation, hard work, and commitment that have gone into building a platform that exceeds customer expectations.
As industry professionals, we must hold analyst firms accountable for the narratives they shape. Accuracy isn’t just a courtesy; it’s a responsibility. A report that gets it wrong doesn’t just harm one brand—it undermines the credibility of the entire industry.
A Call for Accountability
We’re sharing this not to point fingers, but to highlight the importance of fairness and accuracy in evaluations. Analyst firms have a duty to ensure their reports are based on verifiable facts, not assumptions or outdated information. When presented with clear evidence, it’s their responsibility to reflect those corrections.
For our part, we remain committed to transparency and innovation. We encourage anyone evaluating Strobes to reach out directly to us for the full picture. Let’s ensure that decisions are made based on facts, not inaccuracies.
Conclusion
While we respect the effort that goes into creating comprehensive market reports, accuracy and fairness are essential for guiding organizations in their decision-making. The discrepancies in the GigaOm Radar Report highlight the importance of digging deeper and understanding the full picture.
While analyst reports provide valuable market insights, it’s crucial for organizations to look beyond these reports and conduct their own technical evaluations. The cybersecurity industry’s rapid evolution demands that we prioritize actual capabilities and innovation over market presence and analyst relationships.
For startups and emerging vendors, the challenge is clear: technical excellence alone isn’t enough; you must also navigate the complex landscape of industry analysis and market perception. However, the growing trend toward open evaluation frameworks and community-driven assessments offers hope for more balanced market analysis in the future.
The key takeaway for security leaders and procurement teams is to maintain a balanced approach:
- Consider analyst reports as one data point among many.
- Conduct thorough technical evaluations.
- Seek input from peer organizations.
- Consider emerging solutions alongside established players.
- Focus on solving your specific security challenges.
Only through such a comprehensive approach can we ensure that true innovation is recognized and rewarded in the cybersecurity market.
Strobes is committed to transparency, innovation, and delivering the best possible experience for our customers.
If you’d like to learn more about what sets Strobes apart, feel free to reach out or request a demo today.
