Kari E. Milligan

Legal educators nationwide need to begin teaching students a method for completing Counseling Integrated Question Sets, a novel type of question the National Conference of Bar Examiners (“NCBE”) is introducing on the NextGen bar exam. Counseling Integrated Question Sets require students to answer a series of six multiple choice or short answer questions focused on client counseling or dispute resolution, as they work through an unfolding common fact pattern that also contains rules or elicits rules students have memorized.¹

Students who matriculated to law school in 2023 will need to answer Counseling Integrated Question Sets in order to obtain their law licenses in the at least seven states and territories that will administer the NextGen bar exam in July 2026.² Students who matriculated after 2023 will also need to answer Counseling Integrated Question Sets in order to obtain their law licenses in the numerous other states and territories that will begin rolling out the NextGen bar exam in July 2027 and thereafter.³

To aid in this endeavor, faculty at Mitchell Hamline School of Law developed a five-step method and related tips for completing Counseling Integrated Question Sets that was taught to 358 students at Mitchell Hamline during their first semester of law school in 2024. The goal was to test the method, solicit student feedback, and make responsive refinements.

This article provides readers with the five-step method, an overview of student feedback, and the adjustments made to the method based on that feedback. Appendix A contains a student handout with the relevant method and tips.⁴

I. Five-Step Method & Tips

A. Step One

The first step of the method is to read the call of the first question in the Counseling Integrated Question Set to get a high-level sense of the area of law being tested. If the call of the question does not give the reader a sense of the area of law being tested, the reader is instructed to scan the material preceding the question, which may consist of some facts and law, depending on the question.

The reason for this step is to equip the reader to efficiently analyze the preceding law and facts by enabling the reader to view those law and facts through their preexisting knowledge base on the area of law being tested. For example, if the area of law being tested is the tort of battery, the reader can begin thinking about how the elements and subrules for battery apply to the law and facts in the problem. Even if the reader does not have a preexisting knowledge base on the area of law being tested, knowing they lack such knowledge will beneficially cue the reader to pay special attention to any sources of law provided in the problem.

B. Step Two

The second step is to critically read the: (1) law and then the (2) facts above the first question, taking notes as one reads, and thinking about how the law and facts fit together. Readers are asked to read the law before the facts so they can quickly determine what facts are relevant in light of the law without having to read the facts multiple times. The reason readers are prompted to take notes as they read is to help the reader understand and remember what they read upon the first reading. Further, readers are asked to think about how the law and facts fit together because combining law and facts is a core lawyering skill expected to be required to answer many of the questions in Counseling Integrated Question Sets.

C. Step Three

The third step is to reread the call of the question. This step reminds the reader of the call of the question and creates a natural flow between the step of reading the question and developing an answer.

D. Step Four

The fourth step is to develop an answer to the first question in one’s head if possible. The reason for this instruction is to provide the examinee with dedicated time to analyze the issue and answer the question if possible. If the question is a multiple-choice question, occasionally examinees will be able to answer the question without looking at the answer choices, which are deliberately designed to deceive. Other times, examinees will need to move on to Step Five before developing an answer because they will need the information in the answer choices to do so.

E. Step Five

The fifth step differs based on whether the question type is multiple choice or short answer. If multiple choice, readers are asked to read the answer choices from top to bottom, crossing out or disregarding incorrect answers as they read, and selecting the best answer choice or choices, as instructed. The reason for this step is that examinees often need the information in the answer choices in order to answer the question. Examinees are also encouraged to work through the question, top to bottom, crossing out or disregarding answers as they read to efficiently identify the correct answer choice(s) without skipping around and risking getting lost in the question. Examinees are further instructed to select the best answer choice or choices because, as noted below, the bar examiners may ask examinees to select multiple answer choices.

If short answer, readers are prompted to write sentences, phrases, or words, as instructed, that specifically answer the question. The reason for this step is that examinees can answer Counseling Integrated Question Set short answer questions using brief sentences, clauses, or words and need to avoid crafting lengthy answers in paragraph form, as discussed below.

In conclusion, readers are then told to repeat steps one through five for each question in the Counseling Integrated Question Set, skipping steps that do not apply, until they exhaust the questions.

Below is a chart consolidating the steps:

Step One	Read the call of the first question to get a high-level sense of the area of law being tested. If the call of the question does not give one a sense, scan the preceding material.
Step Two	Critically read the: (1) law and then the (2) facts above the first question, marking them up as one reads, and thinking about how they fit together.
Step Three	Reread the call of the first question.
Step Four	Develop an answer to the first question in one’s head if possible.
Step Five	Multiple-Choice Questions – Read the answer choices top to bottom, crossing out or disregarding incorrect answers as one reads, and selecting the best answer choice or choices, as instructed. Short Answer Questions – Write sentences, phrases, or words, as instructed, that specifically answer the question.
Repeat Steps 1-5 for each question in the Counseling IQS, skipping steps that do not apply, until all questions are exhausted.

F. Related Tips

In addition to the method above, students were given tips for answering Counseling Integrated Question Sets, categorized as follows: (1) The Law, (2) Short Answer Questions, (3) Multiple-Choice Questions, (4) What to Do if You Don’t Know What to Do, and (5) Never Go Backwards.

The Law

Students were given the following tips regarding the law.

Counseling IQSs involve multiple (two or more) areas of law.⁵
Sometimes the examiners will give examinees the law; sometimes examinees will have to use law they have memorized.If the examiners give examinees the law, that law trumps any law examinees have memorized.⁶
Examinees are encouraged to integrate everything they have learned during law school as they think through each question, including about the United States legal system, litigation process, and hierarchy of legal authorities.⁷

Short Answer Questions

Students were given the following tips regarding short answer questions.

Avoid using the “IRAC” methodology to answer the questions; the answers need not be that long.⁸
Be aware that there can be multiple “correct” answers.⁹
Examinees’ answers should be as specific as possible. For example, examinees would want to say, “conduct legal research to locate a state supreme court case defining X,” rather than “look for a case”¹⁰or “the phone call where the client said, ‘okay that’s fine’ was an acceptance,” rather than “there was a contract.”
Be precise with language; be careful not to misstate the law or the facts.

Multiple-Choice Questions

Students were given the following tip regarding multiple-choice questions.

Be careful to select the number of answer choices requested by the examiners (i.e., one, two).¹¹

What to Do if You Don’t Know What to Do

Students were told that if they get stuck on a question, they ought to make an educated guess and move on rather than dwell on the question. Further, they were notified that partial credit is available for questions,¹² so they ought not skip a question simply because they did not know the full answer.

Never Go Backwards

Students were instructed to avoid changing their earlier answers based on information they learned later in the problem.¹³

II. Student Testing & Feedback

The method and related tips above were taught to all of the first-year law school students at Mitchell Hamline in Fall 2024. Students received a handout containing the method and related tips, followed by a lecture on the method and related tips that included a demonstration thereof on an intentional torts Counseling Integrated Question Set developed by a commercial bar exam company and edited by Mitchell Hamline faculty.

Students were then required to complete a second intentional torts Integrated Question Set and answer the following four questions:

Did you use the method you were taught in the Integrated Question Set handout?
On a scale of 1-10, how well did the method work?
Now that you have worked through an Integrated Question Set, what changes would you make to the method?
Now that you have worked through an Integrated Question Set, what additional information do you think the Integrated Question Set handout should contain?

333 students answered the four questions provided above. 307 answered the first question, “Did you use the method you were taught in the Integrated Question Set handout?” in the affirmative. Interestingly, 325 students responded to the question, “On a scale of 1-10, how well did the method work?”, so evidently eighteen students who did not use the method provided feedback about how well the method worked. The vast majority of respondents reported that the method worked well, with 56% of students ranking the method from 7-8.9 and an additional 29% of students ranking the method from 9-10. Less than 1% of students rated the method a four or below.

Figure 1. Student Testing & Feedback – All Students

The results did not differ significantly based on whether the students using the methodology were part-time or full-time. A slightly higher percentage of part-time students rated the method a 9-10 than that of full-time students.

Figure 2. Student Testing & Feedback – Part-Time Students

Figure 3. Student Testing & Feedback – Full-Time Students

Some common themes arose based on students’ answers to the following questions: (1) “Now that you’ve worked through an IQS, what changes would you make to the method?” and (2) “Now that you’ve worked through an IQS, what additional information do you think the IQS handout should contain?”

A. Short Answer Questions

The most dominant theme was a request for more clarity about how to format answers to short answer questions. For example, students wrote:

I would like a little more information about how to craft short answers and the appropriate scope of information to be provided. The handout does contain some of this information, but for students who have spent a long time learning to craft CREAC or IRAC legal arguments, it was hard to know what was enough for this type of exam. – Anonymous Student #1

After working through an IQS, I believe the handout would benefit from further explanation of how to write an answer [to short answer questions] without using the IRAC method, while still containing the relevant and most important information. As there is limited space and time to write the response for the IQS in practice, it would be helpful to receive more information about how to write a complete answer without using the IRAC form, which would take more time than a 1-2 clause answer. – Anonymous Student #2

B. Physical Scratch Paper

A second theme was that students found taking notes on physical scratch paper when completing Counseling Integrated Question Sets helpful and requested greater instruction on notetaking. For example, one student stated that “scratch paper may be helpful for more complicated sets,” and another remarked, “I tried to do one set with scratch paper and one without . . . and the method was easier with paper.” A third student advised: “I would add more emphasis on the physical interaction with the question and handwriting notes. . . I found the hand writing of notes while working to answer the question in my head helped me keep my thoughts organized as I prepared my answers.”

C. Never Go Backwards

A third theme was that some students found the instruction in the handout to “Never Go Backwards” confusing and unhelpful. For example, one student said, “I would [] look into explaining why you shouldn’t go backwards. Is it so that you don’t second guess your answer? Waste time?” Another noted:

I would allow the option to go backwards when answering IQSs. While I understand the logic of not questioning/dwelling, sometimes I remember more helpful points or rules that could improve an answer on a former question, so I would like to go back and [check and] update my answer if there is time to do so.

D. Miscellaneous

Of the remaining student answers, a random assortment appeared uniquely helpful. First, one student wanted the handout to “[s]tress that the bar examiners might be looking for a specific number of answer choices on MCQ[s].” Another did not know if they needed to provide citations.

III. Refinements to the Five-Step Method Based on Student Feedback

The relatively high rankings students accorded to the method suggest that the method does not need substantial revisions. However, minor adjustments to the method and tips were made in response to the themes raised in the student answers, each discussed in turn below.

A. Short Answer Questions

To address students’ request for more clarity about how to format responses to short answer questions, the following tip was added to the tip section of the handout: Each short answer question will give examinees the suggested length (e.g., number of sentences),¹⁴ format (e.g., sentences, phrases, or words),¹⁵ and the number of answer fields in which examinees will provide their answers (e.g., a question asking for three facts will contain three answer fields).¹⁶

To provide students with additional guidance about how to format responses to short answer questions, legal educators will likely require further examples of Counseling Integrated Question Sets from the NCBE. To date, this author is only aware of three Counseling Integrated Question Sets released by the NCBE – one to the general public¹⁷ and two to licensees.¹⁸

B. Physical Scratch Paper

Based on this study, student consensus is that physical scratch paper is helpful and ought to be available. However, it is not yet known whether examiners will provide NextGen bar examinees with physical scratch paper, so the language in the method and tips reflects this reality.

C. Never Go Backwards

To address student concern that the instruction in the handout to “Never Go Backwards” was confusing and unhelpful, the relevant tip was deleted in its entirety and replaced with the following:

“Beware of Going Backwards

a. While information provided earlier in the set may be used to answer questions later in the set, information provided later in the set should never be used to answer questions earlier in the set.¹⁹

b. If examinees have extra time and check their earlier answers, they ought to be careful not to talk themselves out of a correct answer.”

D. Miscellaneous

To address remaining student feedback, the author underscored that the bar examiners, on multiple-choice questions, might ask students to select multiple answer choices, and noted in the tip section of the handout that citations are not required for short answer responses.²⁰

IV. Further Research

This article invites further testing of the method and tips for students in their final semester of law school to see if these students provide different feedback than their 1L counterparts due to their increased knowledge and experience.²¹ This article also invites testing of the method and tips on further Counseling Integrated Question Sets released by the NCBE. Additionally, this article invites study into whether the method and tips improve students’ NextGen bar exam scores or scores on Counseling Integrated Question Sets administered during law school. Finally, this article invites development and testing of a method and tips for Drafting Integrated Question Sets, the additional type of Integrated Question Set that will be on the NextGen bar exam, which “focuse[s] on drafting or editing a legal document.”²²

V. Methodological Reflections

The process used in this study of developing a method for performing a legal task such as Counseling Integrated Question Sets, and testing that method by soliciting student feedback, and making responsive refinements (“develop – solicit student feedback – refine”) is well-suited to a range of methods taught by legal educators, including methods for critical reading, case briefing, outlining, and answering essay questions.

This process is valuable because students are well-equipped to provide relevant and important feedback to legal educators. Students who are high enough academic performers to reach law school have, even if implicitly, spent years using different methods to achieve various educational objectives and presumably altering methods that yielded poor academic results.

That said, the utility of student feedback can be limited by an array of factors. For example, in this study, the students’ perception of the method’s efficacy may have been skewed by their lack of experience. The students in this study were in their first semester of law school and had limited experience completing legal problems like Counseling Integrated Question Sets.

Further, the ultimate measure of a methods’ efficacy arguably ought to be students’ performance scores, a metric for which the “develop – solicit student feedback – refine” process does not necessarily account. In fact, in this study, students did not receive performance scores on their Counseling Integrated Question Set answers, but rather self-reflected based on representative answers.

On balance, however, the “develop – solicit student feedback – refine” process still provides legal educators with relevant and important information and has at least three distinct benefits over assessing a method’s efficacy using students’ performance scores. First, collecting and analyzing student feedback is often more feasible for legal educators than collecting and analyzing quantitative data like performance scores. This is particularly true when, as is so often the case, analysis of quantitative data requires the administration of sophisticated statistical techniques such as regression analyses. Second, the process of collecting and analyzing student feedback over student performance scores has the advantage of yielding students’ qualitative suggestions for improving any given method. Third, students themselves receive a unique benefit when asked to provide feedback on a method in the form of improving their metacognitive skills.²³ While collecting student feedback may be slightly more difficult than collecting performance scores, any added burden is modest due to the fact that many legal educators are well-equipped to solicit feedback from their students (captive survey participants) through technological tools such as Canvas and SurveyMonkey.

VI. Conclusion

In conclusion, this article provides legal educators with a student-tested, student-approved method for completing Counseling Integrated Question Sets, as well as avenues of further study to advance NextGen skills-based pedagogy and, ultimately, student achievement.

Appendix A. Integrated Question Set Handout

*Note the content below will change as the National Conference of Bar Examiners (“NCBE”) continues to release more information about Integrated Question Sets.

Overview of Integrated Question Sets

Integrated Question Sets (“IQSs”) are a type of question the NCBE is introducing on the NextGen bar exam. According to the NCBE, “just under a quarter of the NextGen bar exam will be devoted to IQSs. Each of these sets is based on a common fact scenario and may include some legal resources (e.g., excerpts of statutes or judicial opinions) and/or supplemental documents (e.g., a police report or excerpt from a deposition) . . . In addition to testing doctrinal law, some IQSs will focus on counseling and/or dispute resolution [(‘Counseling IQSs’)]; other sets will focus on drafting or editing a legal document [(‘Drafting IQSs’)].” This handout addresses Counseling IQSs.

You will have 6 IQSs on the NextGen bar exam.

You will have 24 minutes to complete each IQS.

Description of Counseling Integrated Question Sets

Counseling Integrated Question Sets are focused on what you as an attorney need to do to serve a fictitious client.

Counseling IQSs contain 6 questions and include a mixture of multiple-choice questions (“MCQs”) and short answer questions.

Method for Completing Integrated Question Sets

Read the call of the first question to get a high-level sense of the area of law being tested. If the call of the question does not give you a sense, scan the preceding material.
Critically read the: (1) law and then the (2) facts above the first question, marking them up as you go, and thinking about how they fit together.
Reread the call of the first question.
Develop an answer to the first question in your head if possible.
For:

Multiple-Choice Questions

Read the answer choices top to bottom, crossing out or disregarding incorrect answers as you read, and selecting the best answer choice or choices, as instructed.

Short Answer Questions

Write sentences, phrases, or words, as instructed, that specifically answer the question.

Repeat Steps 1-5 for each question in the Counseling IQS, skipping steps that do not apply, until you exhaust the questions.

Tips for Completing Counseling Integrated Question Sets

The Law

Counseling IQSs involve multiple (two or more) areas of law.

Sometimes the bar examiners will give you the law; sometimes you will have to use law you have memorized.

If the bar examiners give you the law, that law trumps any law you have memorized.

Integrate everything you have learned during law school as you think through the question, including about the United States legal system, litigation process, and hierarchy of legal authorities.

Short Answer Questions

You do not need to IRAC your answer or use the “Legal Conclusion because Facts, so explanation if needed” sentence structure.

Each short answer question will give you the suggested length (e.g., number of sentences), format (e.g., sentences, phrases, or words), and the number of answer fields in which you should provide your answers (e.g., a question asking for three facts will contain three answer fields).

Be aware that there can be multiple “correct” answers.

Your answers should be as specific as possible. For example, you want to say: “conduct legal research to locate a state supreme court case defining X” rather than “look for a case”, and“ the phone call where the client said ‘okay that’s fine’ was an acceptance” rather than “there was a contract.”

Be precise with your language; be careful not to misstate the law or the facts.

You do not need to provide citations.

Multiple-Choice Questions

The bar examiners may ask you to select multiple answer choices (i.e., one, two); be careful to select the number requested.

What to Do if You Don’t Know What to Do

If you get stuck on a question, make an educated guess and move on (do not dwell).

Partial credit is available for questions; do not skip a question just because you do not know the full answer.

Beware of Going Backwards

While information provided earlier in the set may be used to answer questions later in the set, information provided later in the set should never be used to answer questions earlier in the set.

If you have extra time and check your earlier answers, be careful not to talk yourself out of a correct answer.

Kari E. Milligan is a Teaching Professor of Law at Mitchell Hamline School of Law (“Mitchell Hamline”) in St. Paul, Minnesota. The author is deeply indebted to her colleagues in the Office of Academic Excellence at Mitchell Hamline, who provided invaluable contributions with respect to the methodology in this article and taught the methodology to Mitchell Hamline students while the author was a Visiting Professor of Law at the University of St. Thomas in Minneapolis, Minnesota. The author is also deeply indebted to Cassie Christopher, Mike Gianelloni, and Julie Oseid, who provided insightful feedback on this article.

^{↑ 1}
NextGen (July 2026), Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/exams/nextgen (last visited Dec. 6, 2024) [https://perma.cc/66UK-PGZH].
^{↑ 2}
Id.
^{↑ 3}
See infra pp. 19–21.
^{↑ 4}
National Conference of Bar Examiners, NextGen Study Aid Licensing, Vimeo (Apr. 8, 2024), https://vimeo.com/932200377/4c0f2beeed?share=copy [https://perma.cc/K7E3-GC6T].
^{↑ 5}
Id.
^{↑ 6}
This tip was contributed by Academic Excellence faculty at Mitchell Hamline based on their experience preparing graduates for the bar exam.
^{↑ 7}
E-mail from Mike Gianelloni, Managing Ed., National Conference of Bar Examiners, to Kari Milligan, Teaching Prof. of L., Mitchell Hamline Sch. of L. (Dec. 4, 2024, 09:48 CST) (on file with author). Graduates will likely use the IRAC methodology to complete Performance Task questions on the NextGen bar exam, and variations of the methodology throughout their legal careers. See Sample NextGen Bar Exam Performance Task, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/exams/nextgen/sample-questions/performance-task (last visited Dec. 6, 2024) (prompting students to “draft a short memo” that does not contain a statement of facts but incorporates the law and the relevant facts to reach a reasoned conclusion) [https://perma.cc/BQS5-G5NW].
^{↑ 8}
National Conference of Bar Examiners, supra note 5.
^{↑ 9}
This and the following tip were contributed by Academic Excellence faculty at Mitchell Hamline based on their experience preparing graduates for the bar exam.
^{↑ 10}
Nat’l Conf. of Bar Exam’rs, supra note 1.
^{↑ 11}
National Conference of Bar Examiners, NextGen Practice Set 1 2 (2024) (unpublished question set) (on file with author).
^{↑ 12}
Nat’l Conf. of Bar Exam’rs, supra note 1 (encouraging avoiding “hinging,” or using information gained from a later question to perceive a relationship with an earlier question).
^{↑ 13}
See National Conference of Bar Examiners, NextGen Practice Set 1 8 (2024) (Integrated Question Set 1, Question 2 of 6; “The length of each answer should be about two sentences.”) (unpublished question set) (on file with author).
^{↑ 14}
See id.
^{↑ 15}
See id. at 2.
^{↑ 16}
Nat’l Conf. of Bar Exam’rs, supra note 1.
^{↑ 17}
Id. See also E-mail from Lu Riese, Licensing Program Manager, Nat’l Conf. of Bar Exam’rs. to Kari Milligan, Teaching Prof. of L., Mitchell Hamline Sch. of L. (Sept. 13, 2024, 17:02) (on file with author).
^{↑ 18}
Nat’l Conf. of Bar Exam’rs, supra note 1.
^{↑ 19}
E-mail from Mike Gianelloni, Managing Ed., National Conference of Bar Examiners, to Kari Milligan, Teaching Prof. of L., Mitchell Hamline Sch. of L. (Dec. 4, 2024, 9:48) (on file with author).
^{↑ 20}
Mitchell Hamline faculty did not test the method and tips on students in their final semester of law school in 2023–24 because these students will take the traditional bar exam that does not include Integrated Question Sets.
^{↑ 21}
Nat’l Conf. of Bar Exam’rs, supra note 1.
^{↑ 22}
Metacognition is the skill of “thinking about [one’s] thinking,” and “self-regulat[ing] . . . learning with the goal of transferring learned skills to new situations.” Anthony Niedwiecki, Teaching for Lifelong Learning: Improving the Metacognitive Skills of Law Students Through More Effective Formative Assessment Techniques, 40 Cap. U. L. Rev. 149, 156 (2012).
^{↑ 23}
Sample NextGen Bar Exam Integrated Question Sets, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/exams/nextgen/sample-questions/integrated-question-sets (last visited Nov. 22, 2024) [https://perma.cc/7MCM-MJM6]. Additionally, the NextGen bar exam will contain a second new question type not addressed in this article, Drafting Integrated Question Sets, which task students with drafting or editing a legal document. Id.