Market evaluation

1.Calculation method

Evaluator

At the developer's request, generative AI (Claude Code) calculated the costs from public market rates and work hours.

Basis

The calculation is based on public statistics and market rates (an approximate functional size method, productivity, phase ratios, unit rates, and reduction rates). Each value used has a source, and values without one are marked as assumptions. The in-house cost is also calculated with public statistical rates, not actual salaries.

As of

Based on the work hours and structural figures recorded as of 2026-09-17, calculated on 2026-09-27.

Note

This is not an assessment by a third-party organization. It covers two perspectives, technical and market, that the project overview slides do not.

This report compares estimates for developing a system equivalent to Matsuba with external teams against the actual development by one developer who used generative AI (Claude Code) in every phase. The external side has three scenarios, a contract development company, freelancers, and an in-house SaaS company, and together with the in-house (actual) scenario, the four scenarios are shown under two conditions each: without generative AI and with generative AI.

The estimating method differs by scenario. For the three external scenarios, the functional size (an indicative function point count) is derived from the database structure at the reference point and divided by the productivity in public statistics to estimate the total effort, which is then allocated to requirements definition, design through integration test, and system test using the phase ratio in public statistics. The design-through-integration-test portion is split into the five phases of foundation, database, backend, frontend, and cloud in proportion to the actual hours of those phases. The in-house (actual) scenario uses the recorded work hours, and its without-AI figures are not observed values but estimates back-calculated from the actual hours using the per-phase reduction rates.

The three external scenarios share the same effort, duration, headcount, and role allocation; the only difference is the source of the unit rates. Duration follows the standard duration formula (a coefficient multiplied by the cube root of person-months), and headcount is the average number of people working at once, person-months divided by duration.

All unit rates are aligned to before-tax values; where a source gives a range the median is used, for statistics the average across all company sizes and both sexes is used, and fractions of a yen are rounded. Hours per person-month follow the definition of a public institution, and the monthly actual hours of the information and communications industry in public statistics were confirmed to be close to it^[1][2]. Generative AI subscriptions are counted at list price only for the with-AI condition of the in-house (actual) scenario and are not added to the external scenarios (adding them would raise the external costs, so leaving them out is the conservative choice).

The indicative function point count is a rough estimate based only on the number of data groups, and the productivity in public statistics reflects contract development before generative AI. The limits of these assumptions are summarized in the note in the appendix.

The values and sources used in the calculation are listed in the following table.

Back to contents

2.Size and effort estimate

The functional size was derived from the database structure at the reference point using the NESMA indicative method. The number of user-recognizable data groups (logical files) is 17, and the number of data groups maintained by external systems and only referenced by this system (external interface files) is 1; multiplying the former by 35 FP and the latter by 15 FP gives a functional size of 610 FP.

Dividing this by the productivity for new development in public statistics, 0.151 FP per person-hour, gives a total effort without generative AI of 4,040 hours. Allocating this to the phases using the phase ratio in public statistics leaves the total unchanged: the external effort is 4,040 hours (equivalent to 25.3 person-months), and with generative AI, applying the per-phase reduction rates gives 3,431 hours.

The in-house (actual) effort is the 727 hours in the work hours record, and the development period is 4.1 months. The without-AI figure is back-calculated phase by phase from the actual hours using the per-phase reduction rates, giving 885 hours (the reduction rate weighted by the actual hours averages 17.1%, and applying this average once gives a slightly different total). Deriving effort from lines of code with public productivity statistics is not used, because the lines of code generated by generative AI cannot be compared with statistics of handwritten code.

Back to contents

3.Conclusion

Without generative AI, the cost of the contract development company is 7.3× the actual cost. With generative AI, it is 6.2× the actual cost. Even the lowest of the external scenarios, the in-house SaaS company with generative AI, is 4.4× the actual cost. The costs of the freelancers and the in-house SaaS company apply different unit rates to the same effort, and the only difference among the three external scenarios is the source of the unit rates.

Most of the difference comes from the estimating method. The external effort is an estimate that applies the industry's median productivity to the functional size. That productivity reflects contract development by teams before generative AI and includes coordination, documentation, and review. The in-house figure is the measured work hours of one developer, so the assumptions and the team differ. The rest of the difference comes from the source of the unit rates (outsourcing rates, staffing service rates, and annual pay from public statistics) and from the reduction rates of generative AI. The in-house SaaS company uses the same public statistics on annual pay as the in-house scenario, so its rate difference is small. The in-house cost with generative AI is the labor cost of the actual hours plus the generative AI subscriptions, and the difference from the without-AI figure is the labor cost of the difference between the back-calculated hours and the actual hours, minus the subscriptions.

Back to contents

4.Cost by scenario

4.1Contract development company

This scenario outsources the whole project to a development company with the roles of project manager, architect, engineer, infrastructure engineer, QA engineer, and designer. It includes requirements definition through system test and project management, and excludes the client's preparation of requirements, acceptance testing, maintenance, and cloud usage fees. In general, the back-and-forth of contracts and estimates and the handling of specification changes affect effort and cost. The duration is 9.4 months (an average headcount of 3 people) without generative AI and 8.9 months (an average headcount of 2 people) with generative AI.

4.2Freelancers

This scenario assembles self-employed engineers by job type with the same division of roles as the contract development company. The scope is the same, and project management is handled by the project manager role within the team. The unit rates are averages by job type from published statistics of staffing services, with the QA engineer and designer rates taken as the medians of ranges in articles. The sources do not state whether the listed figures are the freelancer's receipts or the client's payments. In general, dependence on specific individuals and continuity are the concerns.

4.3In-house SaaS company

This scenario is a company with a product development team developing with its own employees, mapping the roles of product manager, tech lead, engineers, SRE, QA, and designer onto the same role allocation. The scope is the same, and the range of personnel costs is as described in the note on the overhead rate. Because the standard phase model is applied, the effort may come out higher than a product team's actual practice.

4.4In-house (actual)

This is the record of one developer who handled everything from requirements analysis to deployment, using generative AI (Claude Code) in every phase. It includes all items in the work hours record and the generative AI subscriptions, and excludes equipment, cloud usage fees, and maintenance. The generative AI subscriptions are the list price × usage period of each seat (account) that the same developer used in parallel. The unit rates are hourly rates derived from annual pay by job type and the overhead rate in public statistics, not actual salaries. The without-AI figures are not observed values but estimates back-calculated from the actual hours using the per-phase reduction rates, and because the conservative end of the sources is used, they are on the lower side.

Back to contents

5.Appendix

5.1Basis of unit rates

The unit rates by scenario and role and their sources are as follows. The contract development company's rate is the weighted average outsourcing rate for scratch development from an industry association survey; it reflects user companies' outsourcing records and is not by job type, so the role column describes the team composition. The freelancer rates are averages by job type from published statistics of staffing services (the QA engineer and designer rates are medians of ranges in articles), and the sources do not state whether the listed figures are the freelancer's receipts or the client's payments. The in-house SaaS company and in-house rates are derived from annual pay by job type in public statistics (the 2025 survey of regular workers, all company sizes, both sexes; a year of regularly paid monthly wages plus annual bonuses and other special payments, before tax), and the in-house rates are not actual salaries. Among the rate sources, only the freelance QA engineer article states that its rates exclude tax; the other industry association and staffing service sources state no tax treatment, so they are treated as before tax.

5.2Reduction with generative AI

5.3Work hours

5.4How the figures are counted

Screens 68

Number of page.tsx files in the web app's App Router tree, excluding the developer preview screens

API endpoints 375

Number of paths in the OpenAPI definition (402 operations by HTTP method)

Database tables 61

Number of model declarations in the Prisma schema

Logical files 17

Number of user-recognizable data groups among the 61 Prisma models that users reference or maintain through screens or the API (internal logical files for the NESMA indicative function point count). Excludes 44 models of 4 kinds: 3 pure many-to-many join tables, 24 technical helper tables (sessions, tokens, job states, and the like), 6 derived or aggregate-only tables, and 11 dependent tables counted together with their parent

External interface files 1

Number of data groups maintained by an external system and only referenced by this system (external interface files for the NESMA indicative function point count): one, the Google Workspace directory (user status). The Google Drive file list, which this system also writes, and operational measurements of the system itself (storage and delivery metrics) are not counted

Work hours 727 hours

Sum of the work hours recorded per phase during the development period (27 items including requirements analysis, design, development, testing, and documentation)

Back to contents

6.Sources

1. [1]

   「ソフトウェア開発データ白書」シリーズに関するよくある質問と回答(opens in a new tab) 独立行政法人情報処理推進機構（IPA）

   https://www.ipa.go.jp/archive/publish/wp-sd/qa.html accessed 2026-09-27

2. [2]

   毎月勤労統計調査 令和7年分結果確報(opens in a new tab) 厚生労働省

   https://www.mhlw.go.jp/toukei/itiran/roudou/monthly/r07/25cr/25cr.html accessed 2026-09-27

3. [3]

   ソフトウェア・メトリクス調査2025【ガイドブック】(opens in a new tab) 日本情報システム・ユーザー協会（JUAS）

   https://juas.or.jp/cms/media/2025/03/25swm.pdf accessed 2026-09-27

4. [4]

   フリーランスエンジニアの月収はいくら？言語別・職種別の平均単価も解説(opens in a new tab) レバテックフリーランス

   https://freelance.levtech.jp/guide/detail/875/ accessed 2026-09-27

5. [5]

   QAエンジニアのフリーランス単価相場｜案件動向と単価アップの条件(opens in a new tab) フリコン

   https://freelance-concierge.jp/articles/detail/435/ accessed 2026-09-27

6. [6]

   フリーランスUIデザイナーの採用で押さえておきたい単価の相場やスキルを解説(opens in a new tab) クロスデザイナー

   https://www.xdesigner.jp/contents/ui-designer-freelance-unit-cost accessed 2026-09-27

7. [7]

   令和7年賃金構造基本統計調査 一般労働者 職種 第1表 職種（小分類）別きまって支給する現金給与額、所定内給与額及び年間賞与その他特別給与額（産業計）(opens in a new tab) 厚生労働省（e-Stat）

   https://www.e-stat.go.jp/stat-search/files?page=1&layout=datalist&toukei=00450091&tstat=000001011429&cycle=0&tclass1=000001229845&tclass2=000001229849&tclass3=000001229855 accessed 2026-09-27

8. [8]

   令和3年就労条件総合調査 結果の概況 労働費用(opens in a new tab) 厚生労働省

   https://www.mhlw.go.jp/toukei/itiran/roudou/jikan/syurou/21/dl/gaiyou03.pdf accessed 2026-09-27

9. [9]

   基準外国為替相場及び裁定外国為替相場（令和8年10月中において適用）(opens in a new tab) 日本銀行

   https://www.boj.or.jp/about/services/tame/tame_rate/kijun/kiju2610.htm accessed 2026-09-27

10. [10]

    Early Function Point Analysis(opens in a new tab) Nesma

    https://assets.zyrosite.com/1rPgtNfl8dfsv1lv/g20-2015-early-function-point-analysis-vs-2015-07-15-en-tM0nQiFhNXVbD6qn.pdf accessed 2026-09-27

11. [11]

    ソフトウェア開発分析データ集2022(opens in a new tab) 独立行政法人情報処理推進機構（IPA）

    https://www.ipa.go.jp/digital/software-survey/metrics/hjuojm000000c6it-att/000102171.pdf accessed 2026-09-27

12. [12]

    What is the Max plan?(opens in a new tab) Anthropic (Claude Help Center)

    https://support.claude.com/en/articles/11049741-what-is-the-max-plan accessed 2026-09-27

13. [13]

    Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality(opens in a new tab) Harvard Business School (Working Paper 24-013)

    https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id4573321.pdf accessed 2026-09-27

14. [14]

    Experimental evidence on the productivity effects of generative artificial intelligence(opens in a new tab) Science (abstract on PubMed)

    https://pubmed.ncbi.nlm.nih.gov/37440646/ accessed 2026-09-27

15. [15]

    How much does AI impact development speed? An enterprise-based randomized controlled trial(opens in a new tab) Google (arXiv)

    https://arxiv.org/abs/2410.12944 accessed 2026-09-27

16. [16]

    Unleashing developer productivity with generative AI(opens in a new tab) McKinsey & Company

    https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/unleashing-developer-productivity-with-generative-ai accessed 2026-09-27

17. [17]

    IaC-Eval: A Code Generation Benchmark for Cloud Infrastructure-as-Code Programs(opens in a new tab) NeurIPS 2024 (Datasets and Benchmarks Track)

    https://neurips.cc/virtual/2024/poster/97835 accessed 2026-09-27

18. [18]

    2025 Developer Survey: AI(opens in a new tab) Stack Overflow

    https://survey.stackoverflow.co/2025/ai accessed 2026-09-27

19. [19]

    Automated Unit Test Improvement using Large Language Models at Meta(opens in a new tab) Meta (arXiv)

    https://arxiv.org/abs/2402.09171 accessed 2026-09-27

20. [20]

    The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study(opens in a new tab) ACM Transactions on Software Engineering and Methodology (arXiv)

    https://arxiv.org/abs/2507.03156 accessed 2026-09-27

21. [21]

    Accelerate State of DevOps Report 2024(opens in a new tab) Google Cloud DORA

    https://dora.dev/research/2024/dora-report/ accessed 2026-09-27

22. [22]

    Generative AI and the Nature of Work(opens in a new tab) Harvard Business School (Working Paper 25-021)

    https://www.hbs.edu/ris/download.aspx?name=25-021.pdf accessed 2026-09-27

23. [23]

    The Impact of AI on Developer Productivity: Evidence from GitHub Copilot(opens in a new tab) Microsoft Research, GitHub, and MIT (arXiv)

    https://arxiv.org/abs/2302.06590 accessed 2026-09-27

24. [24]

    Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity(opens in a new tab) METR

    https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ accessed 2026-09-27

25. [25]

    We are Changing our Developer Productivity Experiment Design(opens in a new tab) METR

    https://metr.org/blog/2026-02-24-uplift-update/ accessed 2026-09-27