ChallengeA Finnish bank (cooperative bank within OP Financial Group's network — 85,000 customers, €4.2 billion in assets) was launching a new digital banking platform — mobile app and web banking — to replace its ageing internet banking service. The platform: built by an external development team (20 developers, 12 months of development). The bank's internal QA capability: 1 QA engineer (manual testing) who had: tested the legacy system. The new platform: fundamentally different technology (React Native mobile, React web, Node.js APIs, microservices architecture) requiring: a fundamentally different testing approach. Testing gaps: no automated testing (the development team had written: minimal unit tests (12% code coverage). No integration tests. No end-to-end tests. No performance tests. Manual testing: conducted by developers testing their own code — the worst practice in software testing (developers: unconsciously avoiding: the scenarios most likely to: reveal bugs in their code)), no Finnish-specific testing (the development team: primarily English-speaking — Finnish-language testing limited to: basic visual checks that: Finnish text displayed. No testing of: Finnish-language accuracy (correct financial terminology), Finnish data format handling (henkilötunnus, IBAN, dates), or: Finnish accessibility (screen reader testing with: Finnish text)), no performance testing (the platform: never load-tested. Expected production load: 15,000 concurrent users during peak (Monday mornings — salary recipients checking: balances and paying bills). The bank: unable to confirm: the platform could handle: this load without: degradation or: failure), and no security testing (the platform: handling sensitive financial data — account balances, transaction history, personal identity numbers. No penetration testing. No security code review. No OWASP vulnerability scanning. The bank's CISO: refusing to approve: production launch without: comprehensive security testing). FIN-FSA requirement: the bank needed to demonstrate: testing adequacy as part of: the regulatory notification for: the new digital channel. The regulator expecting: documented test strategy, test results, and: evidence of: performance and security testing.
SolutionWe established a comprehensive testing programme over 8 weeks — testing the platform while: providing the documentation needed for: FIN-FSA notification. Test automation (weeks 1-4): building the automated test foundation. End-to-end test suite (Playwright): 340 automated test scenarios covering: every customer-facing function — login (biometric and password), account overview (all account types — current, savings, loan, investment), domestic transfer (SEPA), international transfer (SWIFT), bill payment (single and batch), standing orders, card management, loan information, investment overview, and: customer service. Each test: executed in Finnish (the tests interacting with: Finnish-language interfaces, entering: Finnish-format data, and: verifying: Finnish-language output). API test suite: 580 API test cases covering: every backend API endpoint — positive scenarios (correct requests producing: correct responses), negative scenarios (invalid requests producing: appropriate error responses), edge cases (boundary values, empty fields, maximum lengths), and: security scenarios (authentication required, authorisation enforced, rate limiting functional). The API tests: verifying: Finnish data handling (henkilötunnus passed correctly through: all API layers, IBAN validated correctly, ALV calculations correct for: investment transactions)). Finnish-specific testing (weeks 2-5): verifying Finnish banking quality. Finnish language review (native Finnish-speaking financial testers: reviewing every screen, every notification, every email, and: every error message for: correct Finnish financial terminology ("tili" not "account", "viitemaksu" not "reference payment"), grammatically correct Finnish (including: the morphological variations that: many Finnish UIs get wrong — "tililtä" (from the account) not: "tilista"), and: appropriate register (formal Finnish for: legal text, clear Finnish for: instructions, and: friendly Finnish for: notifications)). Finnish data testing (comprehensive testing of: Finnish-specific data handling — henkilötunnus: every validation rule tested (format, check digit, date extraction, age calculation, temporary numbers). Finnish IBAN: format validation, check digit validation, and: BIC lookup from IBAN. Finnish phone numbers: +358 format, 0-prefixed domestic format, and: mobile number identification. Finnish addresses: postinumero lookup, municipality mapping, and: special characters in street names (ä, ö in: "Hämeentie", "Töölönkatu")). Payment testing: Finnish domestic payments tested against: SEPA payment standards — correct IBAN formatting, correct BIC resolution, reference number (viite) validation (Finnish reference number check digit algorithm), and: payment timing (same-day SEPA, next-day, and: scheduled). Performance testing (weeks 3-6): ensuring the platform handles Finnish banking loads. Load profile definition: analysing the legacy system's actual usage patterns — daily transaction volumes, hourly distribution, Monday morning peak, month-end salary period, and: seasonal patterns (Christmas, summer holiday). The load profile: replicating: actual Finnish banking behaviour, not: generic load patterns. Load testing execution: simulating up to: 20,000 concurrent users (133% of expected peak — providing: headroom). Results: the platform handling: 15,000 concurrent users with: sub-2-second response times for: all transactions. At 18,000 users: response times degrading to: 3-4 seconds for: complex operations (international transfers, investment transactions) — acceptable but: identified as: the scaling threshold. At 20,000 users: specific API services approaching: resource limits — identified and: optimised (database query optimisation and: caching improvements reducing: the bottleneck). Stress testing: pushing beyond: expected peaks to: identify failure modes. The platform: degrading gracefully under extreme load (slowing down rather than: crashing) — an important characteristic for: banking systems where: a crash could: lock customers out of: their accounts. Security testing (weeks 3-7): identifying and remediating vulnerabilities. OWASP assessment: testing against: OWASP Top 10 — SQL injection, XSS, CSRF, broken authentication, security misconfiguration, and: other common vulnerabilities. Results: 3 medium-severity findings (insufficient rate limiting on: login attempts, missing security headers on: 2 API endpoints, and: an information disclosure in: error messages) and: 8 low-severity findings — all: remediated within 2 weeks. Penetration testing: simulating: an attacker targeting the banking platform — attempting: unauthorised access, session hijacking, API abuse, and: data exfiltration. Results: no high-severity findings. The authentication and authorisation: robust. One medium finding: a session that wasn't properly invalidated on: password change — remediated. Secure code review: reviewing: authentication flows, payment processing code, and: data encryption implementation for: security vulnerabilities. Findings: 2 issues in: error handling (potential information leakage in: stack traces) — remediated. Accessibility testing (weeks 5-7): WCAG 2.1 AA compliance. Automated scanning (axe): identifying: 45 issues (colour contrast: 12, missing labels: 8, missing alt text: 15, keyboard focus: 10). Manual testing: identifying: 28 additional issues (keyboard trap in: date picker, screen reader announcement missing: for dynamic content updates, focus management: incorrect after: modal dialogs, and: touch target size: too small for: 3 buttons on mobile). All issues: documented, prioritised, and: remediated before launch. Finnish screen reader testing: testing with: NVDA reading Finnish content — verifying: Finnish financial amounts announced correctly ("viisikymmentäkaksituhatta euroa ja neljäkymmentäviisi senttiä" for: €52,000.45), Finnish date formats announced correctly, and: Finnish navigation labels: meaningful and: correctly ordered.
OutcomeThe testing programme completed in 8 weeks. Test coverage: from 12% code coverage to: 78% code coverage. Automated test count: 920 (340 end-to-end + 580 API). Test execution time: full regression in 45 minutes (automated) vs: 5 days (previous manual approach). Defects found: 184 defects identified during the testing programme. Severity: 12 critical (payment processing errors, authentication issues), 34 high (incorrect calculations, data display errors), 78 medium (UI issues, error handling gaps), 60 low (cosmetic, minor). All critical and high: remediated before launch. Medium: remediated within 4 weeks post-launch. Finnish-specific defects: 38 (21% of total) — issues that: non-Finnish testers would not have found — incorrect Finnish financial terminology (7), Finnish data format errors (12), Finnish character display issues (8), Finnish sorting errors (4), and: Finnish accessibility issues (7). These: would have been: customer-facing bugs that: eroded trust in: a market where: Finnish-language quality matters. Performance results: platform handling 15,000 concurrent users within: 2-second response time SLA. Performance improvements: 3 bottlenecks identified and: optimised during testing — the platform's peak capacity: increased from 14,000 to 22,000 concurrent users. Security results: 0 high-severity vulnerabilities. 4 medium: all remediated. 8 low: all remediated. The CISO: approving production launch. Penetration test: passed. FIN-FSA notification: the testing documentation — test strategy, test results, performance evidence, security evidence — submitted as part of: the regulatory notification. FIN-FSA: no additional questions about testing — the documentation: meeting their expectations. Launch outcome: the platform launched on schedule — the first week: no critical production incidents. Post-launch defect rate: 3 bugs reported by customers in the first month (all low-severity UI issues) — compared to: the bank's previous platform launch (5 years ago): 47 bugs reported in the first month, including: 5 that affected: payment processing. Financial impact: defect prevention (184 defects found pre-launch × estimated €2,500 average cost per production defect): €460,000 in avoided costs. Performance assurance (avoided: production performance incidents that would have: locked out customers and: generated: FIN-FSA concern): estimated €200,000 in avoided risk. Security assurance (avoided: potential security incidents): estimated €500,000 in avoided risk. Testing automation (ongoing: 45-minute regression vs: 5-day manual — enabling: weekly releases instead of: quarterly): estimated €180,000 annually in: testing efficiency and: faster feature delivery. Total estimated value: €1.34 million. Testing programme cost: €68,000. ROI: 19.7x.