Justifying Your Data Analysis Method: What Reviewers Look For and What They Reject
Naming a technique isn't enough—reviewers want an argument for fit. This article explains why analysis sections fail and how to justify your method by tying it to your research questions and gap.

Justifying Your Data Analysis Method: What Reviewers Look For and What They Reject academic research methodology data analysis
Reviewers look for a clear justification of why your analysis choices fit your research question and the gap in the literature, not just a description of the techniques. They reject plans that are under-specified, misaligned, or lack contingency for data that do not fit.
You have a solid research question, a literature review that identifies a real gap, and a methods chapter that describes your sample and instruments. Then you reach the data analysis section. You list the tests you ran, the coding approach you used, the software you opened. Your supervisor reads it and says, 'But why these? Why not something else?' That question is where many otherwise strong proposals and dissertations stall. This article is about how to answer it.
Why does the data analysis section fail when it only names techniques?
A data analysis section that reads like a menu of methods tells the reader what you did. It leaves out why that was the right thing to do. Reviewers see this constantly: a paragraph that announces 'interviews were transcribed and thematically analysed using Braun and Clarke's six-phase approach' and then moves on. The technique is named, but the reasoning is absent. The reviewer is left to infer why thematic analysis suited the research question, why that version of it was chosen over alternatives, and how the analysis would produce findings that address the gap.
Naming techniques feels safe because it is concrete. You can point to a procedure and say you followed it. Justification is harder because it requires you to make an argument about fit: fit between question and method, method and data, data and claim. That argument is what separates a defensible analysis from a procedural one. A procedural analysis can be replicated. A justified one can be replicated and evaluated.
The failure is not usually a lack of technical knowledge. Many candidates know their methods well. The failure is rhetorical. They have not been taught to write about analysis as a series of reasoned choices. In a proposal, the analysis section often gets a fraction of the space devoted to the literature review, and the space it gets is spent on procedure. In a dissertation, the analysis chapter can become a travelogue of what was done, with little reflection on why each decision was made and what alternatives were rejected.
Reviewers are not asking for a defence of the entire philosophy of social science. They are asking for evidence that you thought about alternatives and can explain your choice in terms of the study's aims. When that evidence is missing, the section reads as arbitrary, and arbitrary analysis is easy to reject.
How do you tie the analysis plan to the research questions and the gap?
The connection between analysis and research question is often assumed rather than argued. A strong analysis section makes the link explicit. Take a research question like 'How do early-career teachers in rural schools make sense of their professional identity?' The word 'make sense' signals an interpretive interest. An analysis plan that proposes a survey with Likert-scale items about identity would be misaligned, no matter how sophisticated the statistical model. The reviewer will spot the mismatch immediately.
To tie analysis to question, work backwards from the kind of claim you want to make. If you want to claim that a policy was implemented differently across sites, you need an analysis that can compare sites and account for context. If you want to claim that a particular group experiences a phenomenon in a certain way, you need an analysis that can capture experience and meaning. The analysis plan should state the kind of claim and then show that the chosen method can support it.
The gap in the literature adds another layer. Suppose the gap is that existing studies of teacher identity rely on cross-sectional surveys and say little about how identity shifts over the first three years. Your analysis plan should not just use interviews; it should use a longitudinal design and an analysis approach suited to change over time, such as repeated narrative interviews analysed with a diachronic lens. The reviewer should be able to see that the analysis is designed to fill the specific gap, not just to generate data.
A practical way to test alignment is to write a sentence for each research question: 'To answer RQ1, I will use X because it allows me to [specific analytic goal], which is necessary because the literature currently [gap].' If you cannot complete the because clause without repeating the method name, the alignment is weak.
Finally, alignment is not just about the main method. It runs through sampling, data collection, and analysis. A proposal that uses purposive sampling to select information-rich cases but then analyses the data with a technique that aggregates across cases will look inconsistent. Reviewers notice these seams.
What does rigour look like in qualitative, quantitative, and mixed-methods analysis?
Rigour is a word that gets used as if it meant the same thing everywhere. It does not. In quantitative analysis, rigour is often about validity, reliability, and generalisability. Reviewers look for evidence that measures behave as intended, that assumptions of statistical tests are checked, that missing data are handled transparently, and that conclusions do not exceed what the data can support. A regression table without diagnostics is a red flag. A claim of causality from a cross-sectional design is a red flag. Rigour here is about making the analytic process auditable and the inferential limits clear.
In qualitative analysis, rigour is often framed through trustworthiness: credibility, transferability, dependability, and confirmability. Reviewers want to see how you will ensure that your interpretations are grounded in the data and not just plausible stories. That might mean keeping an audit trail, using multiple coders or peer debriefing, engaging in negative case analysis, or returning to participants for member checking. The specific techniques matter less than the logic: how will you convince a sceptical reader that your themes are not idiosyncratic?
Mixed methods introduces a different kind of rigour: integration. It is not enough to do a survey and some interviews; the analysis must show how the two strands inform each other. Reviewers look for a clear rationale for mixing, a point of interface, and an analysis that addresses divergence as well as convergence. If the qualitative findings contradict the quantitative ones, the analysis should treat that as data, not as a problem to be smoothed over.
Across all three, rigour involves transparency about decisions. That means documenting analytic choices as they are made, not reconstructing them after the fact. It means acknowledging the role of the researcher in qualitative work and the role of assumptions in quantitative work. And it means showing that you can distinguish between what the data show and what you infer from them.
One common mistake is to treat rigour as a checklist. A proposal that says 'I will ensure rigour by using triangulation, member checking, and an audit trail' without explaining how these will be applied to the specific study reads as ritual. Reviewers want to see that the rigour strategies are tailored to the analytic challenges the study will face.
How much detail about analysis belongs in a proposal versus a final dissertation?
A proposal is a plan, and its analysis section is necessarily provisional. The level of detail should be enough to convince a reviewer that you know what you are doing and have thought through the alternatives, but not so much that you pretend to know findings you do not have. In a proposal, you describe the intended analytic approach, justify it, and outline the steps you will take. You might include a draft coding frame or a table of planned analyses, but you should not present results.
The balance shifts in a dissertation. There, the analysis section (often a chapter) must account for what actually happened. That means describing the analytic process as it unfolded, including the dead ends, the revisions to the coding frame, and the decisions made when data did not fit. The dissertation analysis chapter should be more detailed about procedure because it is reporting on work done, but it should still be organised around justification. Reviewers of final dissertations often complain about analysis chapters that are either too thin (just presenting results without explaining how they were derived) or too long on procedure and short on reasoning.
A useful distinction is between prospective and retrospective justification. In a proposal, you justify why this analysis is appropriate for the question and gap. In a dissertation, you justify why this analysis, as actually conducted, supports the claims you make. The retrospective justification should acknowledge what changed and why. If you planned to use a particular statistical test but switched to another because assumptions were violated, the dissertation should say so and justify the switch.
Another difference is the treatment of literature. In a proposal, the analysis section can refer back to the literature review to justify methodological choices. In a dissertation, the analysis chapter should not re-review the literature; it should connect findings to the literature in the discussion. The analysis chapter's job is to show how the findings were produced, not to interpret them in light of prior work.
Finally, the proposal should include a timeline for analysis. Reviewers want to see that you have allocated realistic time for transcription, coding, statistical modelling, and writing. A proposal that says analysis will happen in the final month is a warning sign.
What do you do when the data do not fit the planned analysis?
Data rarely behave as planned. Missing values appear, outliers distort distributions, interviews take unexpected turns, and themes emerge that were not anticipated. The analysis plan should include a contingency paragraph that says what you will do if the data do not fit. Writing that paragraph shows forethought, and reviewers read it that way.
For quantitative data, missing data is a common issue. The plan should specify how missingness will be assessed (e.g., is it random or systematic?) and what methods will be used to handle it (e.g., multiple imputation, full information maximum likelihood, or deletion with justification). Outliers should be defined and treated according to a stated rule, not ad hoc. If the sample size falls short of what a planned test requires, the plan should say what alternative will be used or how additional data will be sought. Reviewers look for awareness that these decisions affect results.
For qualitative data, unexpected themes are not a problem to be eliminated. They often turn out to be the most interesting findings in the study. The analysis plan should allow for emergent coding while still maintaining a clear audit trail. If the data do not support the planned analytic framework, the plan should say how the framework will be adapted. For example, if you planned a deductive analysis based on a theoretical model but participants talk about issues outside the model, you might combine deductive and inductive coding. The key is to justify the adaptation in terms of the research question, not just convenience.
In mixed methods, divergence between strands is a common form of 'data not fitting'. The plan should say how divergence will be analysed and reported. Will you treat it as a finding, a measurement problem, or a signal that the initial model was incomplete? Each response has different implications, and the reviewer will want to see that you have considered them.
A contingency plan should also address practical constraints. If access to a particular dataset is uncertain, what is the fallback? If transcription is delayed, how will analysis be adjusted? These are not hypothetical worries; they are the realities of research, and reviewers know it.
Forcing data into the planned analysis and hiding the mismatch is the response that does the most damage. Reviewers can often tell when this has happened, and it undermines trust in the entire study. Document the mismatch instead, and justify the adaptation in the text.
How do you pre-empt the most common reviewer objections to an analysis plan?
Reviewers tend to raise the same objections across proposals and dissertations. Knowing them in advance lets you address them before they become reasons for rejection.
One common objection is that the analysis is under-specified. The reviewer cannot tell what you will actually do. Pre-empt this by describing the analytic steps in enough detail that someone else could follow them, without turning the section into a textbook chapter. For qualitative analysis, that might mean naming the coding approach, describing how codes will be generated and refined, and explaining how themes will be developed. For quantitative analysis, it means specifying the models, the variables, and the assumptions to be checked.
Another objection is that the analysis is over-specified, leaving no room for discovery. This is especially common in qualitative proposals that present a rigid coding frame before data collection. Pre-empt it by explaining how the frame will be used flexibly and how emergent codes will be handled.
A third objection is that the analysis does not match the research question. This is the alignment issue discussed earlier. Pre-empt it by making the link explicit in the text and by showing that alternative approaches were considered and rejected for stated reasons.
A fourth objection is that the analysis is not feasible given the sample or data. Reviewers may question whether you have enough cases for the planned statistical test or whether your access will yield the kind of data needed for the analysis. Pre-empt this with a feasibility paragraph that addresses sample size, data quality, and access.
A fifth objection is that the analysis ignores the role of the researcher. In qualitative work, reviewers want to see reflexivity. Pre-empt this by describing how your positionality might shape the analysis and what steps you will take to mitigate bias.
A sixth objection is that the analysis is not ethical. This might involve privacy in data handling or the potential for harm in interpretation. Pre-empt it by referring to your ethics approval and describing how data will be stored and analysed securely.
Finally, reviewers often object that the analysis section is disconnected from the literature review. They want to see that the analysis is designed to address the gap, not just to apply a method. Pre-empt this by explicitly linking the analysis to the gap in the literature and by showing how the findings will speak to the theoretical framework.
Write the section as though a sceptical but fair reader is sitting across from you. Anticipate their questions and answer them in the text. That is what justification means.
How do you write the analysis section so it connects to the literature review and methods without repeating them?
The analysis section sits between the literature review and the methods, and it can easily become a summary of both. The literature review establishes the gap; the methods describe how data were collected; the analysis should explain how the data will be turned into findings that address the gap. The link between them is conceptual.
One way to achieve this is to use the gap as an organising device. For each research question, state what the literature currently says (or fails to say) and then describe the analysis that will produce the missing knowledge. This keeps the focus on the contribution rather than on the procedure.
Another way is to refer back to the methods without restating them. Instead of saying 'I will conduct semi-structured interviews with 30 teachers', say 'The interviews described in Chapter 3 will be analysed using...' This avoids duplication and keeps the analysis section focused on its own work.
Similarly, refer to the literature review by concept, not by author. Instead of summarising Smith (2018) again, say 'The framework developed in the literature review suggests that identity is negotiated across three domains; the analysis will therefore examine how participants talk about each domain.' This shows integration without repetition.
The analysis section should also connect to the theoretical framework. If the study is informed by a particular theory, the analysis should show how the theory will be operationalised. For example, if you are using Bourdieu's concept of habitus, the analysis might describe how habitus will be identified in interview talk or observed practices. Here the theory is being put to work analytically, and the section should show how.
Finally, the analysis section should point forward to the findings. In a proposal, this means stating what kind of findings the analysis is expected to produce and how they will be presented. In a dissertation, it means providing a bridge to the results chapter. The reader should finish the analysis section knowing what to expect next and why it matters.
Writing the analysis section well requires you to hold three things in mind at once: the question, the data, and the claim. The question tells you what you need to know; the data tells you what you have to work with; the claim tells you what you will be able to say. The justification is the argument that connects them, and without it the section stalls no matter how much technical detail it carries.
Frequently Asked Questions
How long should the data analysis section be in a proposal?
There is no fixed length, but it should be long enough to justify your choices and address likely objections. In many proposals, two to four pages is typical, depending on the complexity of the analysis. Focus on quality of argument rather than word count.
Do I need to include statistical formulae or code in my analysis plan?
Usually not. Reviewers want to see that you understand the method and can justify it, not that you can reproduce the manual. You might mention software and key procedures, but detailed code belongs in an appendix if at all.
What if my supervisor disagrees with my choice of analysis method?
Discuss the rationale for your choice and ask for specific concerns. If the disagreement is about fit with the research question, revisit the alignment. If it is about feasibility, consider alternatives. The goal is a defensible plan, not a personal preference.
Can I change my analysis plan after the proposal is approved?
Yes, but you should document the change and justify it in the dissertation. Changes are common when data do not fit the plan. What matters is that the change is reasoned and transparent, and that you can explain why the original plan was not followed.
How do I demonstrate rigour in qualitative analysis without using quantitative language?
Use the language of trustworthiness and describe concrete strategies: audit trail, peer debriefing, negative case analysis, member checking. Explain how each strategy addresses a specific threat to the credibility of your interpretations.
What is the most common mistake in analysis sections?
Describing what was done without explaining why it was appropriate. Reviewers want to see the reasoning behind your choices, including why alternatives were rejected.
How do I handle missing data in a proposal?
State how you will assess missingness, what methods you will use to handle it, and why those methods are appropriate for your data and question. Show that you understand the assumptions involved.
Should I include a contingency plan in my analysis section?
Yes. A brief paragraph on what you will do if data do not fit the plan shows forethought and can pre-empt reviewer concerns about feasibility.
The notes on the literature review cover the next question, so this page does not repeat them.
The notes on Braun and Clarke's six-phase approach cover the next question, so this page does not repeat them.
The notes on PhD Admission Requirements: Committees cover the next question, so this page does not repeat them.