an essay about social sciences

The Critical Turkey

Essay Writing Hacks for the Social Sciences

What Should Be in a Social Science Essay? Fundamentals and Essential Techniques

This blogpost is also available as a PDF download , so it can be stored on your desktop and used as a checklist before submitting your essay.

The following is a condensed overview of the most important features of social science essay writing. Its aim is to cut through the noise, and focus on the most essential (and important) elements of essay writing. Read it carefully, and use it as a check-list once you have completed your essay.

Before we get into the details, however, be aware: The purpose of writing essays in the social and political sciences is not so much to just demonstrate your knowledge. Rather, it is about applying this knowledge, using it to make a well-informed, well-reasoned, independently-reflected argument that is based on verified (and verifiable) evidence. What should be in an essay, and how you should write it, is all informed by this purpose.

What’s in an Essay?

The main focus of an academic essay, article or book is to address a research or essay question. Therefore, make sure you have read the essay question carefully, think about what aspects of the topic you need to address, and organize the essay accordingly. Your essay should have three parts:

Introduction
Provide context to the question. Be specific (not ‘since the dawn of time, social scientists have been arguing…’, but ‘one of the key debates in the study of revolutions revolves around…’, ideally providing references to the key authors of said debate).
It is almost always a good idea to formulate an argument – an arguable statement – in relation to the essay question (e.g. if the question is ‘Evaluate Weber and Marx’s accounts of capitalism’, an argument could be ‘I am going to argue that Weber is most insightful on X, but Marx is important for Y’). This builds a nice critical element into your essay, your own take on things, going beyond merely describing what others have written.
Essay plan: Tell the reader about the points you are going to cover, and the order in which you are going to do this (e.g. ‘First, the essay looks at…, second… third…’ etc.). Think of it as a roadmap to the essay.
Define key concepts as necessary for understanding. Do not use general dictionaries, as they often contain notions that social scientists try to challenge. Use definitions from the readings, and from sociological dictionaries.
Length: Intro should be between 5 to 10%, and no more than about 10 per cent of the overall word count.
Main Part / Body
The structure of the essay body is informed by the research/essay question: What points do you need to include in order to address the question? What sub-questions are there to the big question? Concentrate on the ‘need-to-knows’ rather than the ‘nice-to-knows’ .
The order in which you arrange these points depends on what makes the most convincing line of argument. This depends on the essay question, but as a rule of thumb you want to build up your argument, from the basics to the more elaborate points, from the weaker to the stronger, from what contradicts your argument to what supports it.
The different points should be addressed in appropriate depth. Make sure you explain not just what something is, but also how it works, and use examples and illustration.
There should be a coherent thread running through the essay and connecting the various points to one another and the overall argument. Indicate these connections in strategic places with appropriate signposting. These signpostings should also help you develop your argument as you proceed.
Excellent essays often raise counter-arguments to the argument presented, and then provide arguments against those counter-arguments. Think about why and how someone might disagree about what you are saying, and how you would respond to them.
Use peer-reviewed academic sources and present evidence for the points you make, using references, reliable statistics, examples etc. Any opinion you express should be built on reliable evidence and good reasoning.
What, finally, is your answer to the question? Bring the various strings of the essay together, summarize them briefly in the context of the essay question, and round off by connecting to the bigger discussion that the essay question is part of. It is usually a good idea to have a differentiated conclusion, in which you e.g. agree with a statement to a certain extent or under specific circumstances (and explain which and why), but disagree with some other aspects of it, rather than making undifferentiated black-or-white statements. You can also contextualise your argument with your ideas from the introduction. It is normally not a good idea to introduce new material in the conclusion. You are wrapping up here, and rounding off, not starting new discussions.
Conclusion should be about, and no longer than, 10 per cent of the overall word count.

Notes on Writing Style

Find the right balance between formal and informal. Avoid being too informal and conversational on the one hand. But also don’t use overly convoluted and complicated language, as it makes your writing inaccessible, and can lead to a lack of clarity. You may at times encounter academic writing that seems deliberately obscure or overcomplicated, but those are not examples you should try to emulate.
Clarity and specificity should indeed be a top priority. Are the words you are using expressing what you want to express? Is it clear who specifically is doing what or saying what? Pay attention to this when proofreading the essay. Could someone understand this differently? Avoid ambiguities.
Key concepts should be clearly defined and used throughout the essay in the way you defined them. Choose the definitions that are most useful for your discussion.
Avoid hyperbole (don’t do ‘shocking statistics’ or ‘dire consequences’ etc.).

Notes on the Writing Process

Proofreading: When you are first writing, don’t think of it as the final product, but treat it as a first draft. Go through several drafts until you are happy with it. At a minimum, proofread the entire essay once or twice. Don’t be perfectionist when you start out, as you can always come back and improve on whatever you’ve written.
Small steps: Focussing on the small, concrete steps of your writing process rather than constantly thinking of the big task at hand will help you feel in control.
Procrastination: Feeling overwhelmed, as well as being too perfectionist, are among the leading causes for procrastination. The two previous points should therefore help you address this issue as well. Don’t be too harsh on yourself when you do procrastinate – almost everyone does it to some extent .
Over the years, keep addressing areas you want to improve on, and keep looking for information. Search online, for example ‘how to cite a book chapter in Harvard Sage’, ‘developing an argument’, ‘ using quotations ’, ‘memory techniques’, ‘how to read with speed’, ‘understanding procrastination’, or ‘ what does peer-reviewed mean ’. There is plenty of information, and some seriously good advice out there. See what works for you. Read the feedback you get on your writing, and incorporate it into your next essay.

Final Thoughts

Essay Writing skills are good skills to have in any situation (except maybe in a zombie apocalypse). They will make the studying process easier over time, and hopefully also more fun. But in a wider sense, they are general skills of critical engagement with the world around you, and will help you filter and prioritise the overload of information you are confronted with on an everyday basis. In that sense, they might actually even be helpful in a zombie apocalypse.

Report this page

To report inappropriate content on this page, please use the form below. Upon receiving your report, we will be in touch as per the Take Down Policy of the service.

Please note that personal data collected through this form is used and stored for the purposes of processing this report and communication with you.

If you are unable to report a concern about content via this form please contact the Service Owner .

A Short Handbook for writing essays in the Humanities and Social Sciences

(8 reviews)

Dan Allosso, Bemidji State University

Salvatore F. Allosso

Publisher: Minnesota Libraries Publishing Project

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Aaron Lefkovitz, Professor, City Colleges of Chicago on 5/4/22

Comprehensiveness rating: 5 see less

This writing manual the author wrote and used for decades at the University of California, Davis is very comprehensive. It reviews multiple aspects of how to get started with writing, such as analyzing texts and taking notes, discovering a topic, preparing for discussion, creating a thesis, ordering evidence, building an argument, coherent paragraphs, effective sentences, appropriate words, revising, and a revision checklist. Also, it has a valuable appendix and even references to such philosophers as Aristotle, so it is comprehensive in both a practical and theoretical sense.

Content Accuracy rating: 5

The content of A Short Handbook for Writing Essays in the Humanities and Social Sciences is accurate, error-free, and unbiased. This can be read in the “Analyzing Texts, Taking Notes” section, where the author begins with unbiased, clear questions, such as “what is a text?” The author then goes on to quote from such sources as author W.H. Auden (1907-1973), English-born poet and man of letters who achieved early fame in the 1930s as a hero of the left during the Great Depression.

Relevance/Longevity rating: 5

A Short Handbook for Writing Essays in the Humanities and Social Sciences does represent relevance and longevity, in the sense that its chapters can be carried down from one generation to another without much variation, for example read in the “Discovering a Topic, Preparing for Discussion” chapter. Here, the author mentions that texts students work with at the college level of their education are mostly givens, as far as English classics, History primary and secondary sources that are important to understand a particular event and period, as well as the ways texts were chosen out of multiple texts in that they fit together and lead to a particular place.

Clarity rating: 5

There is a great deal of clarity in A Short Handbook for Writing Essays in the Humanities and Social Sciences, as this text makes difficult subjects easier to understand for most students, simplifying such potentially daunting topics as “creating a thesis”. In this chapter, the author asks a variety of questions, including what interpretation is the author trying to persuade the reader is valid, what are the reasons for this interpretation, how is the interpretation different from other interpretations, and what part of the text will be examined and emphasized, as well as what are the author’s assumptions and potential objections. These clear questions provide a sense of clarity for the reader and add to the text’s strengths.

Consistency rating: 5

This text is very consistent. Each chapter starts with an interesting quote that frames the chapter narrative in a compelling way. Then, the chapters start with very first-hand/direct testimony given to readers who can read the paragraphs in a way that is meant to speak to them rather than use jargon and difficult to understand sentences. Chapters follow similar structures in terms of longer paragraphs followed by definitions and clear statements that function to provide additional information with regards to the content and theme of the chapter discussed.

Modularity rating: 5

A Short Handbook for Writing Essays in the Humanities and Social Sciences does have easily and readily divisible sections that are useful to the reader in that they break up the narrative and provide all sorts of additional information in an aesthetically pleasing way that can be assigned at different points within the course. There are not enormous blocks of text without subheadings and the text does not seem to be overly self-referential. Instead, there are all sorts of references and data from disparate sources that provide for an interesting and informative read.

Organization/Structure/Flow rating: 5

This text is full of effective, concise, and clear sentences, and is organized well in terms of the ways chapters are structured, starting with a quote that has a particular relevance to the chapter theme, including boxed reminders that set themselves apart from the general narrative, and including various bullet points and examples from literature.

Interface rating: 5

Everything that I have read in this textbook signals that it is indeed free of any kind of significant interface issues, including navigation problems, distortion of images/charts, and any other display features that may distract or confuse the reader.

Grammatical Errors rating: 5

Additionally, the text seems to be free of grammatical errors even as it does seem to contain some spacing issues but that could be on my computer only.

Cultural Relevance rating: 5

As far as the text’s cultural relevance, it does seem to rely quite heavily on dead White European philosophers, from Aristotle to Wittgenstein, to add to and accentuate a point, however it is not offensive in the sense that it does not go out of its way to denigrate a particular race, ethnicity, or other cultural background.

This text does a fine job of introducing students to basic essay writing in the Humanities and Social Sciences, its brevity functioning as a strength in that it keeps things fairly simple while adding philosophical and historical contexts to stay academic as well as relevant and consistent.

Reviewed by Megan Anderson, Assistant Professor, Limestone University on 12/7/21

With only 9 chapters, this text does not cover every essay writing skill students need, but it does include focus on the higher order elements of writing. read more

Comprehensiveness rating: 3 see less

With only 9 chapters, this text does not cover every essay writing skill students need, but it does include focus on the higher order elements of writing.

Content Accuracy rating: 4

As the title suggests, the content is sparse, but it appears accurate.

While there are various theories on the teaching of writing in terms of pedagogy, writing skills do not really change in the sense of timeliness. The examples used are relatively common references from history and literature.

Clarity rating: 4

Again the content is minimal, but the material is written in a clear, easy-to-understand manner that would work for even first-year students.

The text is consistent in terms of terminology and framework, and even tone.

Modularity rating: 3

Each chapter is very short so they are easily assignable. And while there are headers, each chapter appears as one long page. Splitting up the content into just a few pages and spacing out the material a little more would be preferential.

Organization/Structure/Flow rating: 2

While I believe that the text covers many of the essential elements of writing, the chapters appear out of order to me. I would have the chapter on "Effective Sentences" before the one on "Coherent Paragraphs". It is also problematic to me to have a chapter on "Ordering Evidence, Building an Argument" listed before the chapters on basic writing components.

Interface rating: 3

The interface could be easier to navigate. There is no next button to move to easily move from chapter to chapter and to access the nine chapters, you have to click on a plus sign linked to what is called "I. Main Body". It is not very difficult to figure out, but it is just not as thought out as it could be. Like having a Roman Numeral I without a Roman Numeral II is a little odd. It also cannot be saved as a printable PDF.

I do not see any grammatical issues.

Cultural Relevance rating: 3

While I do not see any references that are culturally insensitive, there is also no real attempt at diversity or inclusion. Examples are really all from canonical texts, meaning white male authors, like Shakespeare, Hemingway, and Melville.

Reviewed by Anthony Accardi Jr, Adjunct Professor, Middlesex Community College on 5/30/21

In the text "A Short Handbook for Writing Essays in the Humanities and Social Sciences" by Salvatore and Dan Allosso the authors present a simple, easy to follow guide for students to use when organizing, planning, researching, and writing an essay. In addition to essay structure, the authors also provide help with the “basics of effective writing”, including paragraph writing, sentence writing and avoiding common grammatical errors.

The concise format of the text requires that the author’s stay “right on point” which they do effectively and accurately.

By following each step outlined in this text, a student would undoubtedly improve his/her essay writing skills. Each topic the authors address is relevant to the development of a good essay. The strong emphasis put on the steps for writing an essay make this text a guide students will surely refer to again and again throughout their academic careers.

The conversational style used by the authors makes this text easy to read and understand. Most students find writing a nerve-racking ordeal. The authors deal with this by using straightforward language to explain concepts and reinforce the explanations with simple, easy to understand examples.

The authors have designed a textbook consistent from chapter to chapter and "as a whole". In general, each chapter begins with a quote from a famous author about writing, followed by an explanation of the chapter’s topic, followed by a working example. The authors' down to earth writing style is consistent in every chapter of the text.

The short length of the chapters makes them ideal to be read as individual assignments and their compartmentalized structure is suited well for associated writing assignments.

The structure of this text is one of its strongest points. The authors have organized the chapters in a logical order that students should follow when writing an essay.

The text interface is easy to navigate with no issues noticed.

The text is free of grammatical and syntactic errors.

Cultural Relevance rating: 4

The authors have created a text that shows an awareness of the need for cultural sensitivity and is inoffensive and completely class appropriate. . The Chapter titled “Appropriate Words” touches on avoiding the use of “Sexist Language”, which indicates concern for gender respect. Improvement could be made by using a more diverse group of authors for the opening chapter quotes.

I think this text is an excellent source for helping students understand the basic steps needed to write a good essay.

Reviewed by Aerie Bernard, Adjunct Faculty, Humanities, College of DuPage on 4/20/21

Comprehensiveness rating: 4 see less

This short text provides an approachable primer for novice essayists and reminder of standard practices of academic writing for more experienced writers. Rather than go into great depth, the chapters briefly outline the process of writing academic essays at the high school or undergraduate level. The text is comprehensive in that it is organized linearly to guide the writer from taking notes and developing a thesis through writing drafts and revisions. There is no index or glossary provided; however, the table of contents and short chapters ensure that the text is easy to navigate.

The content is accurate and error-free. The text is written by educators who attempt to address what they identify as common errors in student writing. As such, the bias present reflects a preference for standard English and traditional structure in academic writing.

Because the text focuses on standard practices in academic writing such as writing strong topic sentences, creating arguable theses, and avoiding passive voice, I doubt the text could become obsolete anytime soon. The text speaks to current trends in academic writing by including tips such as how to use gender neutral language and gears itself towards the high school and undergraduate level by modeling and promoting the use of a conversational tone in academic writing. Short, well-organized, worksheet-like chapters allow plenty of room for one to add to, update, or adapt this text.

The authors advise student essayists to use language and style that illustrates “genuine human conversation.” The text successfully models a balance of accuracy of language with a conversational tone. It is a pleasant read.

The text is consistent in its use of terminology, framework, and voice.

Short chapters with limited scope provide introductions and jumping off points for further discussions and activities related to academic writing in the humanities and social sciences.

The chapters are arranged to illustrate a start to finish approach to writing essays. Each chapter focuses on an element of essay writing. The organization is clear and logical.

Interface rating: 4

I had no difficulty accessing or reading the text online with my laptop and my phone. I was not as successful viewing the EPUB as a download to my phone. The text was too small in Bluefire reader and the app would not allow viewing at a larger font size. I do not know if the limitation was due to the EPUB or the reader app.

I did not notice any grammatical errors.

The text addresses the importance of avoiding problematic language in academic writing in the chapter “Appropriate Words” and cautions that writers avoid repetition and wordiness, cliches, jargon, pop culture references, empty words, words with contested meanings, code words, and overextended/mixed/misapplied metaphors. I notice the absence of resources, strategies, and discussions about words relating to race, ethnicity, background, or identity. Also, examples throughout the text are primarily Western, male, and white. Steps towards inclusiveness are present, such as strategies for gender neutral writing, but there is room for improvement.

Reviewed by Sarah Fischer, Assistant Professor, Marymount University on 2/1/21

The book is *short* and useful. It gives excellent advice for how students can and should select strong evidence, how to write effective openings and closings, and discusses many common grammatical errors. However, the book does not spend enough... read more

This book is well-researched and contains no errors (in terms of subject matter, usage, or grammar).

Very relevant, especially because so many books on writing are long, and the longer they are the less likely students are to read them. This book does very impactful work in a very limited number of pages.

Clarity rating: 3

The book is very clear and accessible for professors and advanced students. Many of the examples from English and History papers utilized in the book would be quite difficult for many first-year students to follow, however.

Formatting is quite consistent; terminology is consistently and appropriately used.

Sections of this book are small, easy to understand, and not overwhelming for any level of student to read.

This book's organization flows in a logical way.

I did not notice any interface issues.

I did not notice any grammatical errors in the text.

The authors took care to be culturally sensitive.

A few short exercises at the end of each section, which instructors could create for their classes, would help students practice the book's lessons as sort of intermediary step between reading about possible pitfalls and working on their own to eliminate issues from their writing.

Reviewed by Deirdre Sullivan, Adjunct Faculty, English Department, Berkshire Community College on 6/27/20

A Short Handbook for Writing Essays in the Humanities and Social Sciences by Salvatore Allosso and Dan Allosso is a comprehensive and concise work on how to write good essays on the humanities and the social sciences by clearly defining the definitions of those disciplines. I have not seen many indexes and/or glossaries with these online books, so I don't think it is really necessary for the comprehension of the text.

The content was accurate, without error, and unbiased in its content, syntax, and point of view.

I think the light tone, conversational style, and relevance to all who practice the art of writing is both timely and long-lasting. There is a universal appeal to this approach, and while language is always changing, the rules for written work have more longevity.

The book is written in such a way as to engage even the most reluctant reader into a kind of conspiratorial allegiance on how to approach the art of reading well and writing with lucid accuracy, technical prowess, and enlightened awareness.

The text incorporates terminology into the structure and framework of its chapters with clarity and consistency.

The text is proportionate to reasonable reading and writing assignments. In each chapter, there is a clear way of recognizing and analyzing concepts on writing for use toward student outcomes in a writing course.

This text is logically organized to support and sustain its thesis and the thorough exploration of its guiding elements.

There are no significant interface issues, problems with navigation, or distractions to confuse potential readers.

There are no grammatical errors to my reckoning.

The text is not culturally insensitive or offensive in any way. The book embraced multi-culturalism with quotes, questions, and persuasive argument as to how a writer must be objective, open-minded, and thoroughly engaged in standing by their work.

I really loved the conversational style between authors and readers. This father-son duo has clearly taken delight in sharing their love of the world through the art of writing. I really liked the quotes they chose to support their ideas. Perhaps one day, I will use their book in my composition classes. A truly remarkable discovery!

Reviewed by Dayle Turner, Professor, Leeward Community College on 6/27/20

The text covers fairly well the important considerations of writing essays for humanities and social sciences courses. The authors assert their intention of taking students “step-by-step through the process of writing essays for an upper-level high school class or a college course.” The steps of which they speak include analyzing texts, note-taking, formulating essay topics, creating theses, ordering evidence, building arguments, writing coherent paragraphs, composing effective sentences, using appropriate diction, and revising. The text lacks an index and glossary and the inclusion thereof would certainly strengthen the comprehensiveness of the work.

The content of this text is accurate and the steps covered are mostly applicable for first-year college students and high school juniors and seniors.

Relevance/Longevity rating: 4

The content appears up-to-date. Text is devoid of visual imagery, making it potentially less appealing to contemporary/millennial students, but its structure invites relatively easy updating, and all links were accurate.

The text is mostly clear and provides adequate examples to explain the application of material discussed in each chapter.

Consistency rating: 4

The text's consistency would be excellent if an index and glossary were included.

Modularity rating: 4

This text is organized in such a manner that students can be assigned short readings without having to jump hither and yon between chapters or different parts of the book.

There are nine chapters in the text. They are presented in a logical and purposeful order. Critical reading and note-taking comes first while a revision checklist is available at the end. This makes sense as it is important to provide students with suggestions for information-gathering and revision.

The interface is free of any distracting issues. The text is mostly easy to navigate.

I noticed no grammatical errors.

The text successfully represents a variety of races, ethnicities, and backgrounds. Examples are sensitive and free of stereotypes.

This book would have been beneficial to me as an undergraduate. Most of what it covers are things I had to learn by experience, and the quality of my earliest scholarship would have been much improved with the benefit of these lessons. The text has value as a supplementary or recommended material, particularly for students whose plans include graduate school or writing-intensive professions. Students who are most prepared will get the most out of it, but the text also offers good examples

Reviewed by Matilda (Tillie) Yoder, Librarian, Goshen College on 7/10/19

The scope of this text is very clearly outlined in its title - it aims to guide students through the process of writing essays for humanities and social sciences courses. The Allossos succeed in creating a work that does just that, discussing techniques and strategies for writing well but assuming that readers will have a reasonable familiarity with English grammar. Contents included how to develop ideas, how to formulate effective arguments, how to identify weak points in writing, and how to revise effectively. It is worth noting that the authors are not concerned with formatting, emphasizing the writing process and not the finicky details of citation structure, title page layout, or font size.These issues are easily addressed on a great many websites and reference works; more concerning is that there is no real discussion of plagiarism or how to manage citations and references, which is an important part of any humanities or social sciences essay that requires research and not a single text.

There is no glossary or index for this work, though the table of contents lays out chapter topics very clearly. An index would be quite useful for instructors and students wanting to use the book in its entirety. Similarly, a reference list with links to related works and websites might also be of use for those who would like more in-depth information on particular techniques not elaborated on in this short work.

The content of this guidebook is accurate, although its narrow focus does mean that is not comprehensive (and it does not intend to be). The strategies outlined in it are standard practice and are conveyed succinctly. Quoted authors are all referenced by name but not in any further detail; simple citations for these quotes would model best practices for the students reading the material.

The content of this guidebook is general enough in nature to remain relevant for some time. The examples given throughout the book reference works of classic Western literature or established understandings of history that American schools are likely to continue to teach - Shakespeare’s plays, the history of slave uprisings in the Americas, the Civil War, and Hemingway all feature. Notably, references are only discussed in the context of example passages, and so no knowledge of the events or plots is necessary to understand what the authors are saying.

The Alessos practice what they preach in this instance, writing directly and clearly. Jargon is almost non-existent, and where it does exist it is always defined and explained. Concepts are clearly illustrated with multiple examples and outlined step by step. The overall vocabulary and level of writing is appropriate for students in grade 11 or above.

Key terms are used continually throughout this work; in particular, the authors emphasize the importance of unity, coherence, and emphasis in effective writing. Vocabulary terms are introduced and used consistently, although alternative terms are listed to ensure understanding.

This guide could be easily divided into distinct sections useful for a wide variety of classes throughout the humanities and social sciences. History and English classes would find it particularly relevant, but introductory writing teachers, writing tutors, and academic support offices would also find much that is useful here. The sections on how to construct a thesis and the revision checklist are particularly applicable to me in my work as a writing tutor. I can see myself having students read specific sections of this book depending on what their particular roadblocks to writing are.

The organization of this text is logical, beginning with the process of note-taking and brainstorming, and moving on to persuasive argument building, thesis construction, essay structure, writing, and revision. The revision checklist at the end of the textbook is also organized in such a way that it leads students to look for major issues in their writing before the minor ones.

Overall, the guidebook displayed well and is easy to navigate. There are no images included, and although images are not strictly necessary for this sort of topic, I believe that the text would benefit from some formatting changes. Some of the lists could use better visual clues in their subdivision, and example paragraphs would benefit from being presented in a diagram format where specific portions could be highlighted and remarked on more directly. Unfortunately, this title is not available in PDF format, which would be useful for anyone wanting access to the book without an internet connection. Epub format would be useful as well.

I noticed no grammatical errors or typos in this text.

All references to culture in this text appear in example writing passages. Because of this, no deep understanding of the referenced work or work is needed, because it is the writing and not the content of the passage that is the focus. However, almost all of the references included are focused on classic works concerning Western literature and history (Kafka, Dostoyevsky, Beowulf, etc.). A broadening of examples would be welcome, but as it stands the text is inoffensive and reflects what is taught in many English classes in American schools.

Getting Started Writing
Chapter 1: Analyzing Texts, Taking Notes
Chapter 2: Discovering a Topic, Preparing for Discussion
Chapter 3: Creating a Thesis
Chapter 4: Ordering Evidence, Building an Argument
Chapter 5: Coherent Paragraphs
Chapter 6: Effective Sentences
Chapter 7: Appropriate Words
Chapter 8: Revising
Chapter 9: Revision Checklist

Ancillary Material

About the book.

A retired master teacher of English and Comparative Literature teams up with his son, a History professor, on a new version of the writing manual he wrote and used for decades at the University of California, Davis.

About the Contributors

Dan Allosso , Bemidji State University

Contribute to this Page

Request new password
Create a new account

Good Essay Writing: A Social Sciences Guide

Student resources, on this website, you will find a range of resources corresponding to the topics covered in each chapter. just click on links to the left..

Writing good essays is one of the most challenging aspects of studying in the social sciences. This simple guide provides you with proven approaches and techniques to help turn you into a well-oiled, essay-writing machine.

Good Essay Writing demonstrates how to think critically and formulate your argument as well as offering water-tight structuring tips, referencing advice and a word on those all-too-familiar common worries – all brought to life through real student examples from a range of subjects.

This practical guide is an absolute must for everybody wanting – or needing – to brush up on their essay-writing skills and boost their grades.

Disclaimer:

This website may contain links to both internal and external websites. All links included were active at the time the website was launched. SAGE does not operate these external websites and does not necessarily endorse the views expressed within them. SAGE cannot take responsibility for the changing content or nature of linked sites, as these sites are outside of our control and subject to change without our knowledge. If you do find an inactive link to an external website, please try to locate that website by using a search engine. SAGE will endeavour to update inactive or broken links when possible.

Walden University
Faculty Portal

Common Assignments: Writing in the Social Sciences

Although there may be some differences in writing expectations between disciplines, all writers of scholarly material are required to follow basic writing standards such as writing clear, concise, and grammatically correct sentences; using proper punctuation; and, in all Walden programs, using APA style. When writing in the social sciences, however, students must also be familiar with the goals of the discipline as these inform the discipline’s writing expectations. According to Ragin (1994), the primary goal of social science research is “identifying order in the complexity of social life” (para. 1). Serving the primary goal are the following secondary goals:

Identifying general patterns and relationships
Testing and refining theories
Making predictions
Interpreting culturally and historically significant phenomena
Exploring diversity
Giving voice
Advancing new theories (Ragin, 1994, para. 2)

To accomplish these goals, social scientists examine and explain the behavior of individuals, systems, cultures, communities, and so on (Dartmouth Writing Program, 2005), with the hope of adding to the world’s knowledge of a particular issue. Students in the social sciences should have these goals at the back of their minds when choosing a research topic or crafting an effective research question. Instead of simply restating what is already known, students must think in terms of how they can take a topic a step further. The elements that follow are meant to give students an idea of what is expected of social science writers.

If you have content-specific questions, be sure to ask your instructor. The Writing Center is available to help you present your ideas as effectively as possible.

Because one cannot say everything there is to say about a particular subject, writers in the social sciences present their work from a particular perspective. For instance, one might choose to examine the problem of childhood obesity from a psychological perspective versus a social or environmental perspective. One’s particular contribution, proposition, or argument is commonly referred to as the thesis and, according to Gerring et al. (2009), a good thesis is one that is “ new, true, and significant ” (p. 2). To strengthen their theses, social scientists might consider presenting an argument that goes against what is currently accepted within that field while carefully addressing counterarguments, and adequately explaining why the issue under consideration matters (Gerring et al., 2009). For instance, one might interpret a claim made by a classical theorist differently from the manner in which it is commonly interpreted and expound on the implications of the new interpretation. The thesis is particularly important because readers want to know whether the writer has something new or significant to say about a given topic. Thus, as you review the literature, before writing, it is important to find gaps and creative linkages between ideas with the goal of contributing something worthwhile to an ongoing discussion. In crafting an argument, you must remember that social scientists place a premium on ideas that are well reasoned and based on evidence. For a contribution to be worthwhile, you must read the literature carefully and without bias; doing this will enable you to identify some of the subtle differences in the viewpoints presented by different authors and help you to better identify the gaps in the literature. Because the thesis is essentially the heart of your discussion, it must be argued objectively and persuasively.

In examining a research question, social scientists may present a hypothesis and they may choose to use either qualitative or quantitative methods of inquiry or both. The methods most often used include interviews, case studies, observations, surveys, and so on. The nature of the study should dictate the chosen method. (Do keep in mind that not all your papers will require that you employ the various methods of social science research; many will simply require that you analyze an issue and present a well reasoned argument.) When you write your capstones, however, you will be required to come to terms with the reliability of the methods you choose, the validity of your research questions, and ethical considerations. You will also be required to defend each one of these components. The research process as a whole may include the following: formulation of research question, sampling and measurement, research design, and analysis and recommendations. Keep in mind that your method will have an impact on the credibility of your work, so it is important that your methods are rigorous. Walden offers a series of research methods courses to help students become familiar with research methods in the social sciences.

Organization

Most social science research manuscripts contain the same general organizational elements:

Title

Abstract

Introduction

Literature Review

Methods

Results

Discussion

References

Note that the presentation follows a certain logic: in the introduction one presents the issue under consideration; in the literature review, one presents what is already known about the topic (thus providing a context for the discussion), identifies gaps, and presents one’s approach; in the methods section, one identifies the method used to gather data; in the results and discussion sections, one then presents and explains the results in an objective manner, acknowledging the limitations of the study (American Psychological Association [APA], 2020). One may end with a presentation of the implications of the study and areas upon which other researchers might focus.

For a detailed explanation of typical research paper organization and content, be sure to review Table 3.1 (pp. 77-81) and Table 3.2 (pp. 95-99) of your 7th edition APA manual.

Objectivity

Although social scientists continue to debate whether objectivity is achievable in the social sciences and whether theories really represent objective scientific analyses, they agree that one’s work must be presented as objectively as possible. This does not mean that writers cannot be passionate about their subject; it simply means that social scientists are to think of themselves primarily as observers and they must try to present their findings in a neutral manner, avoiding biases, and acknowledging opposing viewpoints.

It is important to note that instructors expect social science students to master the content of the discipline and to be able to use discipline appropriate language in their writing. Successful writers of social science literature have cultivated the thinking skills that are useful in their discipline and are able to communicate professionally, integrating and incorporating the language of their field as appropriate (Colorado State University, 2011). For instance, if one were writing about how aid impacts the development of less developed countries, it would be important to know and understand the different ways in which aid is defined within the field of development studies.

Colorado State University. (2011). Why assign WID tasks? http://wac.colostate.edu/intro/com6a1.cfm

Gerring, J., Yesnowitz, J., & Bird, S. (2009). General advice on social science writing . https://www.bu.edu/polisci/files/people/faculty/gerring/documents/WritingAdvice.pdf

Ragin, C. (1994). Construction social research: The unity and diversity of method . http://poli.haifa.ac.il/~levi/res/mgsr1.htm

Trochim, W. (2006). Research methods knowledge base . http://www.socialresearchmethods.net/kb/

Didn't find what you need? Email us at [email protected] .

Previous Page: Writing in the Disciplines
Next Page: Collaborative Writing in Business & Management
Office of Student Disability Services

Walden Resources

Departments.

Academic Residencies
Academic Skills
Career Planning and Development
Customer Care Team
Field Experience
Military Services
Student Success Advising
Writing Skills

Centers and Offices

Center for Social Change
Office of Academic Support and Instructional Services
Office of Degree Acceleration
Office of Research and Doctoral Services
Office of Student Affairs

Student Resources

Doctoral Writing Assessment
Form & Style Review
Quick Answers
ScholarWorks
SKIL Courses and Workshops
Walden Bookstore
Walden Catalog & Student Handbook
Student Safety/Title IX
Legal & Consumer Information
Website Terms and Conditions
Cookie Policy
Accessibility
Accreditation
State Authorization
Net Price Calculator
Contact Walden

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing in the Social Sciences

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

In this section

Subsections.

Games & Quizzes
History & Society
Science & Tech
Biographies
Animals & Nature
Geography & Travel
Arts & Culture
On This Day
One Good Fact
New Articles
Lifestyles & Social Issues
Philosophy & Religion
Politics, Law & Government
World History
Health & Medicine
Browse Biographies
Birds, Reptiles & Other Vertebrates
Bugs, Mollusks & Other Invertebrates
Environment
Fossils & Geologic Time
Entertainment & Pop Culture
Sports & Recreation
Visual Arts
Demystified
Image Galleries
Infographics
Top Questions
Britannica Kids
Saving Earth
Space Next 50
Student Center
Introduction & Top Questions

Effects of theology

Effects of the classics and of cartesianism.

Heritage of the Enlightenment
Major themes resulting from democratic and industrial change
New ideologies
New intellectual and philosophical tendencies
Political science
Cultural anthropology
Social psychology
Social statistics and social geography
Science and social science
Social science in the research universities
Early attempts at a science of humanity: Durkheim and Weber
Humanity as a symbolic phenomenon
Identity, will, and the thinking self
Institutions, nations, and civilizations
Applications of the science of humanity: nationalism, economic growth, and mental illness

Where was Émile Durkheim educated?
Where did Émile Durkheim work?
Why is Émile Durkheim significant?

Union Jack flag of Great Britain, united kingdom

social science

Our editors will review what you’ve submitted and determine whether to revise the article.

The University of Queensland - Pressbooks - Introduction to the Social Sciences
Statistics LibreTexts - Theories and Social Science
Economic and Social Research Council - What is social science?
social science - Children's Encyclopedia (Ages 8-11)
social sciences - Student Encyclopedia (Ages 11 and up)
Table Of Contents

What is a social science?

A social science is any branch of academic study or science that deals with human behaviour in its social and cultural aspects. Usually included within the social sciences are cultural (or social) anthropology, sociology, psychology, political science, and economics.

What is the relationship between the terms behavioral science and social science ?

Beginning in the 1950s, the term behavioral sciences was often applied to disciplines categorized as social sciences. Some favored this term because it brought these disciplines closer to some of the sciences, such as physical anthropology, which also deal with human behavior.

Who named the social science discipline of sociology?

Auguste Comte gave the social science of sociology its name and established the new discipline in a systematic fashion.

What is cultural anthropology's relationship to the social sciences?

Cultural anthropology is a branch of the social sciences that deals with the study of culture in all of its aspects and that uses the methods, concepts, and data of archaeology, ethnography and ethnology, folklore, and linguistics.

What was Adolphe Quetelet's contribution to the social sciences?

Adolphe Quetelet was a key figure in the social statistics branch of the social sciences. He was the first person to call attention, in a systematic manner, to the kinds of structured behavior that could be observed and identified only through statistical means.

social science , any branch of academic study or science that deals with human behaviour in its social and cultural aspects. Usually included within the social sciences are cultural (or social) anthropology , sociology , psychology , political science , and economics . The discipline of historiography is regarded by many as a social science, and certain areas of historical study are almost indistinguishable from work done in the social sciences. Most historians, however, consider history as one of the humanities . In the United States , focused programs, such as African-American Studies, Latinx Studies, Women, Gender, and Sexuality Studies, are, as a rule , also included among the social sciences, as are often Latin American Studies and Middle Eastern Studies, while, for instance, French, German, or Italian Studies are commonly associated with humanities. In the past, Sovietology was always considered a social science discipline, in contrast to Russian Studies.

Beginning in the 1950s, the term behavioral sciences was often applied to the disciplines designated as the social sciences. Those who favoured this term did so in part because these disciplines were thus brought closer to some of the sciences, such as physical anthropology and physiological psychology , which also deal with human behaviour.

Strictly speaking, the social sciences, as distinct and recognized academic disciplines, emerged only on the cusp of the 20th century. But one must go back farther in time for the origins of some of their fundamental ideas and objectives. In the largest sense, the origins go all the way back to the ancient Greeks and their rationalist inquiries into human nature , the state , and morality . The heritage of both Greece and Rome is a powerful one in the history of social thought, as it is in other areas of Western society. Very probably, apart from the initial Greek determination to study all things in the spirit of dispassionate and rational inquiry, there would be no social sciences today. True, there have been long periods of time, as during the Western Middle Ages , when the Greek rationalist temper was lacking. But the recovery of this temper, through texts of the great classical philosophers, is the very essence of the Renaissance and the Enlightenment in modern European history. With the Enlightenment, in the 17th and 18th centuries, one may begin.

Heritage of the Middle Ages and the Renaissance

The same impulses that led people in that age to explore Earth , the stellar regions, and the nature of matter led them also to explore the institutions around them: state, economy, religion , morality , and, above all, human nature itself. It was the fragmentation of medieval philosophy and theory, and, with this, the shattering of the medieval worldview that had lain deep in thought until about the 16th century, that was the immediate basis of the rise of the several strands of specialized social thought that were in time to provide the inspiration for the social sciences.

Medieval theology , especially as it appears in St. Thomas Aquinas ’s Summa theologiae (1265/66–1273), contained and fashioned syntheses from ideas about humanity and society—ideas indeed that may be seen to be political, social, economic, anthropological, and geographical in their substance. But it is partly this close relation between medieval theology and ideas of the social sciences that accounts for the different trajectories of the social sciences, on the one hand, and the trajectories of the physical and life sciences, on the other. From the time of the English philosopher Roger Bacon in the 13th century, there were at least some rudiments of physical science that were largely independent of medieval theology and philosophy. Historians of physical science have no difficulty in tracing the continuation of this experimental tradition, primitive and irregular though it was by later standards, throughout the Middle Ages . Side by side with the kinds of experiment made notable by Bacon were impressive changes in technology through the medieval period and then, in striking degree , in the Renaissance . Efforts to improve agricultural productivity; the rising utilization of gunpowder , with consequent development of guns and the problems that they presented in ballistics; growing trade , leading to increased use of ships and improvements in the arts of navigation , including use of telescopes ; and the whole range of such mechanical arts in the Middle Ages and Renaissance as architecture , engineering , optics , and the construction of watches and clocks —all of this put a high premium on a pragmatic and operational understanding of at least the simpler principles of mechanics , physics , astronomy , and, in time, chemistry .

In short, by the time of Copernicus and Galileo in the 16th century, a fairly broad substratum of physical science existed, largely empirical but not without theoretical implications on which the edifice of modern physical science could be built. It is notable that the empirical foundations of physiology were being established in the studies of the human body being conducted in medieval schools of medicine and, as the career of Leonardo da Vinci so resplendently illustrates, among artists of the Renaissance, whose interest in accuracy and detail of painting and sculpture led to their careful studies of human anatomy .

Very different was the beginning of the social sciences. In the first place, the Roman Catholic Church , throughout the Middle Ages and even into the Renaissance and Reformation , was much more attentive to what scholars wrote and thought about the human mind and human behaviour in society than it was toward what was being studied and written in the physical sciences. From the church’s point of view, while it might be important to see to it that thought on the physical world corresponded as far as possible to what Scripture said—witnessed, for example, in the famous questioning of Galileo—it was far more important that such correspondence exist in matters affecting the human mind, spirit, and soul . Nearly all the subjects and questions that would form the bases of the social sciences in later centuries were tightly woven into the fabric of medieval Scholasticism , and it was not easy for even the boldest minds to break this fabric.

Then, when the hold of Scholasticism did begin to wane, two fresh influences, equally powerful, came on the scene to prevent anything comparable to the pragmatic and empirical foundations of the physical sciences from forming in the study of humanity and society. The first was the immense appeal of the Greek classics during the Renaissance, especially those of the philosophers Plato and Aristotle . A great deal of social thought during the Renaissance was little more than gloss or commentary on the Greek classics. One sees this throughout the 15th and 16th centuries.

Second, in the 17th century there appeared the powerful influence of the philosopher René Descartes . Cartesianism , as his philosophy was called, declared that the proper approach to understanding of the world, including humanity and society, was through a few simple, fundamental ideas of reality and, then, rigorous, almost geometrical deduction of more complex ideas and eventually of large, encompassing theories, from these simple ideas, all of which, Descartes insisted, were the stock of common sense—the mind that is common to all human beings at birth. It would be hard to exaggerate the impact of Cartesianism on social and political and moral thought during the century and a half following publication of his Discourse on Method (1637) and his Meditations on First Philosophy (1641). Through the Enlightenment into the later 18th century, the spell of Cartesianism was cast on nearly all those who were concerned with the problems of human nature and human society.

Great amounts of data pertinent to the study of human behaviour were becoming available in the 17th and 18th centuries. The emergence of nationalism and the associated impersonal state carried with it ever growing bureaucracies concerned with gathering information, chiefly for taxation , census , and trade purposes. The voluminous and widely published accounts of the great voyages that had begun in the 15th century, the records of soldiers, explorers, and missionaries who perforce had been brought into often long and close contact with indigenous and other non-Western peoples, provided still another great reservoir of data. Until the beginning of the 19th century, these and other empirical materials were used, if at all, solely for illustrative purposes in the writings of the social philosophers. Just as in the equally important area of the study of life, no philosophical framework as yet existed to allow for an objective and comprehensive interpretation of these empirical materials. Only in physics could this be done at the time.

Organizing Your Social Sciences Research Assignments

Annotated Bibliography
Analyzing a Scholarly Journal Article
Group Presentations
Dealing with Nervousness
Using Visual Aids
Grading Someone Else's Paper
Types of Structured Group Activities
Group Project Survival Skills
Leading a Class Discussion
Multiple Book Review Essay
Reviewing Collected Works
Writing a Case Analysis Paper
Writing a Case Study
About Informed Consent
Writing Field Notes
Writing a Policy Memo
Writing a Reflective Paper
Writing a Research Proposal
Generative AI and Writing
Acknowledgments

Reflective writing is a process of identifying, questioning, and critically evaluating course-based learning opportunities, integrated with your own observations, experiences, impressions, beliefs, assumptions, or biases, and which describes how this process stimulated new or creative understanding about the content of the course.

A reflective paper describes and explains in an introspective, first person narrative, your reactions and feelings about either a specific element of the class [e.g., a required reading; a film shown in class] or more generally how you experienced learning throughout the course. Reflective writing assignments can be in the form of a single paper, essays, portfolios, journals, diaries, or blogs. In some cases, your professor may include a reflective writing assignment as a way to obtain student feedback that helps improve the course, either in the moment or for when the class is taught again.

How to Write a Reflection Paper . Academic Skills, Trent University; Writing a Reflection Paper . Writing Center, Lewis University; Critical Reflection . Writing and Communication Centre, University of Waterloo; Tsingos-Lucas et al. "Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats." American Journal of Pharmaceutical Education 81 (2017): Article 8.

Benefits of Reflective Writing Assignments

As the term implies, a reflective paper involves looking inward at oneself in contemplating and bringing meaning to the relationship between course content and the acquisition of new knowledge . Educational research [Bolton, 2010; Ryan, 2011; Tsingos-Lucas et al., 2017] demonstrates that assigning reflective writing tasks enhances learning because it challenges students to confront their own assumptions, biases, and belief systems around what is being taught in class and, in so doing, stimulate student’s decisions, actions, attitudes, and understanding about themselves as learners and in relation to having mastery over their learning. Reflection assignments are also an opportunity to write in a first person narrative about elements of the course, such as the required readings, separate from the exegetic and analytical prose of academic research papers.

Reflection writing often serves multiple purposes simultaneously. In no particular order, here are some of reasons why professors assign reflection papers:

Enhances learning from previous knowledge and experience in order to improve future decision-making and reasoning in practice . Reflective writing in the applied social sciences enhances decision-making skills and academic performance in ways that can inform professional practice. The act of reflective writing creates self-awareness and understanding of others. This is particularly important in clinical and service-oriented professional settings.
Allows students to make sense of classroom content and overall learning experiences in relation to oneself, others, and the conditions that shaped the content and classroom experiences . Reflective writing places you within the course content in ways that can deepen your understanding of the material. Because reflective thinking can help reveal hidden biases, it can help you critically interrogate moments when you do not like or agree with discussions, readings, or other aspects of the course.
Increases awareness of one’s cognitive abilities and the evidence for these attributes . Reflective writing can break down personal doubts about yourself as a learner and highlight specific abilities that may have been hidden or suppressed due to prior assumptions about the strength of your academic abilities [e.g., reading comprehension; problem-solving skills]. Reflective writing, therefore, can have a positive affective [i.e., emotional] impact on your sense of self-worth.
Applying theoretical knowledge and frameworks to real experiences . Reflective writing can help build a bridge of relevancy between theoretical knowledge and the real world. In so doing, this form of writing can lead to a better understanding of underlying theories and their analytical properties applied to professional practice.
Reveals shortcomings that the reader will identify . Evidence suggests that reflective writing can uncover your own shortcomings as a learner, thereby, creating opportunities to anticipate the responses of your professor may have about the quality of your coursework. This can be particularly productive if the reflective paper is written before final submission of an assignment.
Helps students identify their tacit [a.k.a., implicit] knowledge and possible gaps in that knowledge . Tacit knowledge refers to ways of knowing rooted in lived experience, insight, and intuition rather than formal, codified, categorical, or explicit knowledge. In so doing, reflective writing can stimulate students to question their beliefs about a research problem or an element of the course content beyond positivist modes of understanding and representation.
Encourages students to actively monitor their learning processes over a period of time . On-going reflective writing in journals or blogs, for example, can help you maintain or adapt learning strategies in other contexts. The regular, purposeful act of reflection can facilitate continuous deep thinking about the course content as it evolves and changes throughout the term. This, in turn, can increase your overall confidence as a learner.
Relates a student’s personal experience to a wider perspective . Reflection papers can help you see the big picture associated with the content of a course by forcing you to think about the connections between scholarly content and your lived experiences outside of school. It can provide a macro-level understanding of one’s own experiences in relation to the specifics of what is being taught.
If reflective writing is shared, students can exchange stories about their learning experiences, thereby, creating an opportunity to reevaluate their original assumptions or perspectives . In most cases, reflective writing is only viewed by your professor in order to ensure candid feedback from students. However, occasionally, reflective writing is shared and openly discussed in class. During these discussions, new or different perspectives and alternative approaches to solving problems can be generated that would otherwise be hidden. Sharing student's reflections can also reveal collective patterns of thought and emotions about a particular element of the course.

Bolton, Gillie. Reflective Practice: Writing and Professional Development . London: Sage, 2010; Chang, Bo. "Reflection in Learning." Online Learning 23 (2019), 95-110; Cavilla, Derek. "The Effects of Student Reflection on Academic Performance and Motivation." Sage Open 7 (July-September 2017): 1–13; Culbert, Patrick. “Better Teaching? You Can Write On It “ Liberal Education (February 2022); McCabe, Gavin and Tobias Thejll-Madsen. The Reflection Toolkit . University of Edinburgh; The Purpose of Reflection . Introductory Composition at Purdue University; Practice-based and Reflective Learning . Study Advice Study Guides, University of Reading; Ryan, Mary. "Improving Reflective Writing in Higher Education: A Social Semiotic Perspective." Teaching in Higher Education 16 (2011): 99-111; Tsingos-Lucas et al. "Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats." American Journal of Pharmaceutical Education 81 (2017): Article 8; What Benefits Might Reflective Writing Have for My Students? Writing Across the Curriculum Clearinghouse; Rykkje, Linda. "The Tacit Care Knowledge in Reflective Writing: A Practical Wisdom." International Practice Development Journal 7 (September 2017): Article 5; Using Reflective Writing to Deepen Student Learning . Center for Writing, University of Minnesota.

How to Approach Writing a Reflection Paper

Thinking About Reflective Thinking

Educational theorists have developed numerous models of reflective thinking that your professor may use to frame a reflective writing assignment. These models can help you systematically interpret your learning experiences, thereby ensuring that you ask the right questions and have a clear understanding of what should be covered. A model can also represent the overall structure of a reflective paper. Each model establishes a different approach to reflection and will require you to think about your writing differently. If you are unclear how to fit your writing within a particular reflective model, seek clarification from your professor. There are generally two types of reflective writing assignments, each approached in slightly different ways.

1. Reflective Thinking about Course Readings

This type of reflective writing focuses on thoughtfully thinking about the course readings that underpin how most students acquire new knowledge and understanding about the subject of a course. Reflecting on course readings is often assigned in freshmen-level, interdisciplinary courses where the required readings examine topics viewed from multiple perspectives and, as such, provide different ways of analyzing a topic, issue, event, or phenomenon. The purpose of reflective thinking about course readings in the social and behavioral sciences is to elicit your opinions, beliefs, and feelings about the research and its significance. This type of writing can provide an opportunity to break down key assumptions you may have and, in so doing, reveal potential biases in how you interpret the scholarship.

If you are assigned to reflect on course readings, consider the following methods of analysis as prompts that can help you get started :

Examine carefully the main introductory elements of the reading, including the purpose of the study, the theoretical framework being used to test assumptions, and the research questions being addressed. Think about what ideas stood out to you. Why did they? Were these ideas new to you or familiar in some way based on your own lived experiences or prior knowledge?
Develop your ideas around the readings by asking yourself, what do I know about this topic? Where does my existing knowledge about this topic come from? What are the observations or experiences in my life that influence my understanding of the topic? Do I agree or disagree with the main arguments, recommended course of actions, or conclusions made by the author(s)? Why do I feel this way and what is the basis of these feelings?
Make connections between the text and your own beliefs, opinions, or feelings by considering questions like, how do the readings reinforce my existing ideas or assumptions? How the readings challenge these ideas or assumptions? How does this text help me to better understand this topic or research in ways that motivate me to learn more about this area of study?

2. Reflective Thinking about Course Experiences

This type of reflective writing asks you to critically reflect on locating yourself at the conceptual intersection of theory and practice. The purpose of experiential reflection is to evaluate theories or disciplinary-based analytical models based on your introspective assessment of the relationship between hypothetical thinking and practical reality; it offers a way to consider how your own knowledge and skills fit within professional practice. This type of writing also provides an opportunity to evaluate your decisions and actions, as well as how you managed your subsequent successes and failures, within a specific theoretical framework. As a result, abstract concepts can crystallize and become more relevant to you when considered within your own experiences. This can help you formulate plans for self-improvement as you learn.

If you are assigned to reflect on your experiences, consider the following questions as prompts to help you get started :

Contextualize your reflection in relation to the overarching purpose of the course by asking yourself, what did you hope to learn from this course? What were the learning objectives for the course and how did I fit within each of them? How did these goals relate to the main themes or concepts of the course?
Analyze how you experienced the course by asking yourself, what did I learn from this experience? What did I learn about myself? About working in this area of research and study? About how the course relates to my place in society? What assumptions about the course were supported or refuted?
Think introspectively about the ways you experienced learning during the course by asking yourself, did your learning experiences align with the goals or concepts of the course? Why or why do you not feel this way? What was successful and why do you believe this? What would you do differently and why is this important? How will you prepare for a future experience in this area of study?

NOTE: If you are assigned to write a journal or other type of on-going reflection exercise, a helpful approach is to reflect on your reflections by re-reading what you have already written. In other words, review your previous entries as a way to contextualize your feelings, opinions, or beliefs regarding your overall learning experiences. Over time, this can also help reveal hidden patterns or themes related to how you processed your learning experiences. Consider concluding your reflective journal with a summary of how you felt about your learning experiences at critical junctures throughout the course, then use these to write about how you grew as a student learner and how the act of reflecting helped you gain new understanding about the subject of the course and its content.

ANOTHER NOTE: Regardless of whether you write a reflection paper or a journal, do not focus your writing on the past. The act of reflection is intended to think introspectively about previous learning experiences. However, reflective thinking should document the ways in which you progressed in obtaining new insights and understandings about your growth as a learner that can be carried forward in subsequent coursework or in future professional practice. Your writing should reflect a furtherance of increasing personal autonomy and confidence gained from understanding more about yourself as a learner.

Structure and Writing Style

There are no strict academic rules for writing a reflective paper. Reflective writing may be assigned in any class taught in the social and behavioral sciences and, therefore, requirements for the assignment can vary depending on disciplinary-based models of inquiry and learning. The organization of content can also depend on what your professor wants you to write about or based on the type of reflective model used to frame the writing assignment. Despite these possible variations, below is a basic approach to organizing and writing a good reflective paper, followed by a list of problems to avoid.

Pre-flection

In most cases, it's helpful to begin by thinking about your learning experiences and outline what you want to focus on before you begin to write the paper. This can help you organize your thoughts around what was most important to you and what experiences [good or bad] had the most impact on your learning. As described by the University of Waterloo Writing and Communication Centre, preparing to write a reflective paper involves a process of self-analysis that can help organize your thoughts around significant moments of in-class knowledge discovery.

Using a thesis statement as a guide, note what experiences or course content stood out to you , then place these within the context of your observations, reactions, feelings, and opinions. This will help you develop a rough outline of key moments during the course that reflect your growth as a learner. To identify these moments, pose these questions to yourself: What happened? What was my reaction? What were my expectations and how were they different from what transpired? What did I learn?
Critically think about your learning experiences and the course content . This will help you develop a deeper, more nuanced understanding about why these moments were significant or relevant to you. Use the ideas you formulated during the first stage of reflecting to help you think through these moments from both an academic and personal perspective. From an academic perspective, contemplate how the experience enhanced your understanding of a concept, theory, or skill. Ask yourself, did the experience confirm my previous understanding or challenge it in some way. As a result, did this highlight strengths or gaps in your current knowledge? From a personal perspective, think introspectively about why these experiences mattered, if previous expectations or assumptions were confirmed or refuted, and if this surprised, confused, or unnerved you in some way.
Analyze how these experiences and your reactions to them will shape your future thinking and behavior . Reflection implies looking back, but the most important act of reflective writing is considering how beliefs, assumptions, opinions, and feelings were transformed in ways that better prepare you as a learner in the future. Note how this reflective analysis can lead to actions you will take as a result of your experiences, what you will do differently, and how you will apply what you learned in other courses or in professional practice.

Basic Structure and Writing Style

Reflective Background and Context

The first part of your reflection paper should briefly provide background and context in relation to the content or experiences that stood out to you. Highlight the settings, summarize the key readings, or narrate the experiences in relation to the course objectives. Provide background that sets the stage for your reflection. You do not need to go into great detail, but you should provide enough information for the reader to understand what sources of learning you are writing about [e.g., course readings, field experience, guest lecture, class discussions] and why they were important. This section should end with an explanatory thesis statement that expresses the central ideas of your paper and what you want the readers to know, believe, or understand after they finish reading your paper.

Reflective Interpretation

Drawing from your reflective analysis, this is where you can be personal, critical, and creative in expressing how you felt about the course content and learning experiences and how they influenced or altered your feelings, beliefs, assumptions, or biases about the subject of the course. This section is also where you explore the meaning of these experiences in the context of the course and how you gained an awareness of the connections between these moments and your own prior knowledge.

Guided by your thesis statement, a helpful approach is to interpret your learning throughout the course with a series of specific examples drawn from the course content and your learning experiences. These examples should be arranged in sequential order that illustrate your growth as a learner. Reflecting on each example can be done by: 1) introducing a theme or moment that was meaningful to you, 2) describing your previous position about the learning moment and what you thought about it, 3) explaining how your perspective was challenged and/or changed and why, and 4) introspectively stating your current or new feelings, opinions, or beliefs about that experience in class.

It is important to include specific examples drawn from the course and placed within the context of your assumptions, thoughts, opinions, and feelings. A reflective narrative without specific examples does not provide an effective way for the reader to understand the relationship between the course content and how you grew as a learner.

Reflective Conclusions

The conclusion of your reflective paper should provide a summary of your thoughts, feelings, or opinions regarding what you learned about yourself as a result of taking the course. Here are several ways you can frame your conclusions based on the examples you interpreted and reflected on what they meant to you. Each example would need to be tied to the basic theme [thesis statement] of your reflective background section.

Your reflective conclusions can be described in relation to any expectations you had before taking the class [e.g., “I expected the readings to not be relevant to my own experiences growing up in a rural community, but the research actually helped me see that the challenges of developing my identity as a child of immigrants was not that unusual...”].
Your reflective conclusions can explain how what you learned about yourself will change your actions in the future [e.g., “During a discussion in class about the challenges of helping homeless people, I realized that many of these people hate living on the street but lack the ability to see a way out. This made me realize that I wanted to take more classes in psychology...”].
Your reflective conclusions can describe major insights you experienced a critical junctures during the course and how these moments enhanced how you see yourself as a student learner [e.g., "The guest speaker from the Head Start program made me realize why I wanted to pursue a career in elementary education..."].
Your reflective conclusions can reconfigure or reframe how you will approach professional practice and your understanding of your future career aspirations [e.g.,, "The course changed my perceptions about seeking a career in business finance because it made me realize I want to be more engaged in customer service..."]
Your reflective conclusions can explore any learning you derived from the act of reflecting itself [e.g., “Reflecting on the course readings that described how minority students perceive campus activities helped me identify my own biases about the benefits of those activities in acclimating to campus life...”].

NOTE: The length of a reflective paper in the social sciences is usually less than a traditional research paper. However, don’t assume that writing a reflective paper is easier than writing a research paper. A well-conceived critical reflection paper often requires as much time and effort as a research paper because you must purposeful engage in thinking about your learning in ways that you may not be comfortable with or used to. This is particular true while preparing to write because reflective papers are not as structured as a traditional research paper and, therefore, you have to think deliberately about how you want to organize the paper and what elements of the course you want to reflect upon.

ANOTHER NOTE: Do not limit yourself to using only text in reflecting on your learning. If you believe it would be helpful, consider using creative modes of thought or expression such as, illustrations, photographs, or material objects that reflects an experience related to the subject of the course that was important to you [e.g., like a ticket stub to a renowned speaker on campus]. Whatever non-textual element you include, be sure to describe the object's relevance to your personal relationship to the course content.

Problems to Avoid

A reflective paper is not a “mind dump” . Reflective papers document your personal and emotional experiences and, therefore, they do not conform to rigid structures, or schema, to organize information. However, the paper should not be a disjointed, stream-of-consciousness narrative. Reflective papers are still academic pieces of writing that require organized thought, that use academic language and tone , and that apply intellectually-driven critical thinking to the course content and your learning experiences and their significance.

A reflective paper is not a research paper . If you are asked to reflect on a course reading, the reflection will obviously include some description of the research. However, the goal of reflective writing is not to present extraneous ideas to the reader or to "educate" them about the course. The goal is to share a story about your relationship with the learning objectives of the course. Therefore, unlike research papers, you are expected to write from a first person point of view which includes an introspective examination of your own opinions, feelings, and personal assumptions.

A reflection paper is not a book review . Descriptions of the course readings using your own words is not a reflective paper. Reflective writing should focus on how you understood the implications of and were challenged by the course in relation to your own lived experiences or personal assumptions, combined with explanations of how you grew as a student learner based on this internal dialogue. Remember that you are the central object of the paper, not the research materials.

A reflective paper is not an all-inclusive meditation. Do not try to cover everything. The scope of your paper should be well-defined and limited to your specific opinions, feelings, and beliefs about what you determine to be the most significant content of the course and in relation to the learning that took place. Reflections should be detailed enough to covey what you think is important, but your thoughts should be expressed concisely and coherently [as is true for any academic writing assignment].

Critical Reflection . Writing and Communication Centre, University of Waterloo; Critical Reflection: Journals, Opinions, & Reactions . University Writing Center, Texas A&M University; Connor-Greene, Patricia A. “Making Connections: Evaluating the Effectiveness of Journal Writing in Enhancing Student Learning.” Teaching of Psychology 27 (2000): 44-46; Good vs. Bad Reflection Papers , Franklin University; Dyment, Janet E. and Timothy S. O’Connell. "The Quality of Reflection in Student Journals: A Review of Limiting and Enabling Factors." Innovative Higher Education 35 (2010): 233-244: How to Write a Reflection Paper . Academic Skills, Trent University; Amelia TaraJane House. Reflection Paper . Cordia Harrington Center for Excellence, University of Arkansas; Ramlal, Alana, and Désirée S. Augustin. “Engaging Students in Reflective Writing: An Action Research Project.” Educational Action Research 28 (2020): 518-533; Writing a Reflection Paper . Writing Center, Lewis University; McGuire, Lisa, Kathy Lay, and Jon Peters. “Pedagogy of Reflective Writing in Professional Education.” Journal of the Scholarship of Teaching and Learning (2009): 93-107; Critical Reflection . Writing and Communication Centre, University of Waterloo; How Do I Write Reflectively? Academic Skills Toolkit, University of New South Wales Sydney; Reflective Writing . Skills@Library. University of Leeds; Walling, Anne, Johanna Shapiro, and Terry Ast. “What Makes a Good Reflective Paper?” Family Medicine 45 (2013): 7-12; Williams, Kate, Mary Woolliams, and Jane Spiro. Reflective Writing . 2nd edition. London: Red Globe Press, 2020; Yeh, Hui-Chin, Shih-hsien Yang, Jo Shan Fu, and Yen-Chen Shih. “Developing College Students’ Critical Thinking through Reflective Writing.” Higher Education Research and Development (2022): 1-16.

Writing Tip

Focus on Reflecting, Not on Describing

Minimal time and effort should be spent describing the course content you are asked to reflect upon. The purpose of a reflection assignment is to introspectively contemplate your reactions to and feeling about an element of the course. D eflecting the focus away from your own feelings by concentrating on describing the course content can happen particularly if "talking about yourself" [i.e., reflecting] makes you uncomfortable or it is intimidating. However, the intent of reflective writing is to overcome these inhibitions so as to maximize the benefits of introspectively assessing your learning experiences. Keep in mind that, if it is relevant, your feelings of discomfort could be a part of how you critically reflect on any challenges you had during the course [e.g., you realize this discomfort inhibited your willingness to ask questions during class, it fed into your propensity to procrastinate, or it made it difficult participating in groups].

Writing a Reflection Paper . Writing Center, Lewis University; Reflection Paper . Cordia Harrington Center for Excellence, University of Arkansas.

Another Writing Tip

Helpful Videos about Reflective Writing

These two short videos succinctly describe how to approach a reflective writing assignment. They are produced by the Academic Skills department at the University of Melbourne and the Skills Team of the University of Hull, respectively.

<< Previous: Writing a Policy Memo
Next: Writing a Research Proposal >>
Last Updated: Jun 3, 2024 9:44 AM
URL: https://libguides.usc.edu/writingguide/assignments

Interdisciplinarity

What I Have Learned from Social Science

I’ve spent my adult life in and around social science. Academically through studying psychology and linguistics (alongside philosophy), professionally through working at SAGE for over 30 years and personally through an abiding amateur interest in various fields sometimes expressed in my own writing of books or articles.

In light of my recent election as a Fellow of the Academy of Social Sciences. I’ve been reflecting on what social science has meant to me, and why my interest continues to this day.

These reflections are a quite personal take. They are not meant to be a ‘defence of social science’ or a comprehensive review of its impact in various domains, though when people who aren’t familiar with social science ask me what the point of it is I find myself responding in this kind of vein. It’s a personal view on why I think a social science imagination can benefit us as individuals and improve society more generally, especially at a time of such upheaval and reconfiguration.

The starting point for me is in human psychology, the subject of my undergraduate degree. In my first week in October 1985 as a fresher at Exeter University, I met Steve Reicher, who was assigned as my first-year tutor. Steve was a ‘new blood’ lecturer at the time who had a year earlier published what was to become a seminal article analysing the St Paul’s riots in Bristol in April 1980. Through my encounters and discussions with Steve and other psychologists in the department I learned about certain features of human nature. While I didn’t go quite as far as Steve, who would say ‘the nature of human nature is its capacity to transcend itself’, and while the very idea of human nature is, I realise, contested and felt confusing to me initially, I started to learn how profoundly social that nature was.

While this may sound obvious to many – we are social animals who cooperate and learn from each other, of course – I nevertheless find it hard to see myself that way consistently. And I’ve learned that it’s not just me. While social science shows how our natures are deeply social it also explains why we don’t always see this fact that well. When not looking through a social science lens we (in the West at least) tend to see ourselves and our place in the world as more individual than that, like fish swimming around unaware of the environment in which they are suspended.

It’s not that the idea of the individual is a myth. Rather it is one of many identities, all shaped by historical and cultural forces, which tends in our daily lives to be overly emphasised. We see the figure more easily than the ground along which she walks. For instance, what’s known as the ‘fundamental attribution error’ leads me to look at someone’s behaviour and explain it too quickly in terms of their imagined individual characteristics and ignore the context. So if someone cuts me up in traffic I more easily think ‘selfish!’ rather than ‘maybe there’s an emergency’.

A key value of social science, it seems to me, is to counter-balance that self-image , to help us see the ground as well as we see the figure. We know when it comes to physical health that what we want and what is good for us are not always aligned. Well so it is for the social health of this social animal. Our interests, it seems to me, are best served by a more balanced understanding of human circumstances and contexts, but for all sorts of reasons evolutionists like to explore, we don’t do this as fully as we might. The tendency mentioned above for instance, to see the individual more easily than her circumstances, has deep consequences for the chances of human flourishing – for our attitudes toward each other – if left unchecked.

And this point, the need to see more context, can be extended in various ways. Here are 10 examples of tendencies we have which a social science imagination can and should help us to counter-balance, each of which have moral or political implications for how to organise ourselves and society better. This is not to say that each tendency is a problem in itself, or that we can’t reverse it under certain conditions, it’s that a social science imagination is useful in helping us do just that. 1 I’ve added a reference for each one to help provide a bit more insight for those who are interested. But as I say these reflections are personal and highly selective rather than anything systematic. For that you should talk to the experts! I’ve put these 10 into three broad buckets:

Those tendencies which assume we have more agency, more control over our circumstances, than we do, e.g.:

Judgement over luck. It’s easier, thanks to the ‘just-world hypothesis’ and even the idea of meritocracy to assume people have more responsibility for their outcomes than they generally have. So people who end up worse off in life can be blamed for their individual failure to measure up.
Cure over prevention. It’s easier to say ‘lock ’em up’ and harder to be tough on the causes of crime. The same goes for health interventions. We will typically pay more for treatment rather than preventative measures.
The conscious over the unconscious. It’s easier to focus on explicit thoughts and feelings, and to assume we are rational and objective in our judgments while ignoring the less obvious underlying tendencies such as revealed by studies of unconscious bias.

Then there are those which favour the near over the far, whether in terms of time, space or social categories, such as:

Short term over long term. It’s easier to spend now than to save for a pension. Similarly, we can underrate the significance of climate change for future generations.
The near at hand over the far away. It’s easier to care about the incidence of COVID-19 in our own locale rather than further afield. There’s even evidence of a ‘propinquity effect’ which describes how we find people and things more appealing merely by being physically closer to us.
Us over Them. What’s called ‘ingroup favouritism’ makes it easier to sympathise with people ‘like me’ than the members of an outgroup. The recent surge in political polarisation, from Brexit to the recent US election, bears on this tendency.

We have tendencies to oversimplify, to prefer the status quo and then to generalise, such as when we favour

The dominant over the marginalised. It’s easier to see a tall, white middle class man as an authority figure than almost anybody else!
The vivid example over statistical data . It’s easier to fear terrorism and plane crashes than driving cars. And remember the line often attributed to Stalin, that a single death is a tragedy, while a million deaths are a mere statistic.
Choosing the status quo over alternative explanations. It’s easier to say ‘that’s just how things are’, than this is how they got this way and could be different. Much of what feels immutable is in fact socially constructed.
The simple over the complex. It’s easier to skewer politicians on the journalistic jab of ‘answer the question yes or no’, than to accept a more nuanced response. Many social problems are known as ‘wicked’ and don’t always have right or wrong answers, though hopefully better or worse ones.

It’s a simple list which reveals my starting point in psychology, and others (from sociology, anthropology, political science etc) would choose different examples I’m sure. But I hope it shows that tending to think people have more freedom and agency than they do, or tending to favour the near over the far, or to see the social world as fixed rather than constructed comes easily to us, while hampering the possibilities of human progress in many ways.

A social science imagination helps us put a thumb on the scales to counter-balance those tendencies. This offers possibilities to recalibrate society to better suit our social natures than an individualistic essentialising view will be inclined to do. Meanwhile politicians, media outlets, and more generally people with power and wanting to hold on to it exploit these tendencies; and social science analyses that, too.

Social science has a hard time breaking through because it tends not to offer up easy answers and solutions (see point 10 above). But as one physicist pointed out, it is child’s play to understand theoretical physics compared to understanding child’s play. Understanding molecules offers more law-like generalisations and predictions than understanding people and culture. The problems addressed by social science are complex and often don’t have right or wrong answers, but hopefully offer better or worse ones. And often those answers depend on some mix of different levels of analysis.

The complexity of social science reflects the complexities of humanity at many scales and magnitudes. At a global level, scientists study wars and conflict, trans-national migration, cultures and religions, international cooperation and diplomacy between nations. Zoom into a country and they look at forms of government and how power is gained, how the economy works. Zoom further into policy domains and see social scientists looking at crime, aging, mental health, physical health (obesity, vaccine uptake, physical distancing), education, social care, the use of technology, the nature of work, the media, social cohesion, inequality and social injustice. You’ll find them analysing organizations like companies, political parties, schools, prisons, cities, football clubs, unions and the forms of organization that describe how they work, and don’t work, such as leadership, crowd behaviour, discrimination, power. Zoom in further to see them study interpersonal behaviour whether in groups, teams or relationships. Looking into family systems offers yet more levels of complexity even before turning to individual differences and subjective experiences (of love, loneliness, stress, addiction, emotion, memory, motivation) let alone those who dive into perception, cognition, the unconscious and more.

These levels are intersecting and overlapping as much as we are, and the study of them leads social science to interact with other disciplines, from natural sciences on the one side to humanities on the other.

Of course there’s good and bad, deep and trivial, applied and abstract work in social science as in all fields, and the mechanism of generating scholarship which translates to everyday impact and relevance is complex and sometimes badly broken through the many mixed incentives that come from trying to create academic reputations in higher education settings. As the social scientist Garry Brewer once pithily remarked ‘the world has problems while universities have departments’.

With all that said the cumulative intellectual labour of social scientists across the globe does have a powerful effect over time. And it is particularly satisfying watching Steve Reicher, now at St Andrews, commenting influentially on many of today’s political issues. Many of you will have seen his work on government responses to COVID-19 as part of the behavioural science advisory committee to what we call ‘the other SAGE’ and latterly independent SAGE.

But the moment that struck me most forcibly was after the death of George Floyd and the subsequent protests, one of which was the pulling down of the statue of Edward Colston in Bristol — the same city where the St Paul’s riots occurred 40 years before. Steve commented on how this event did not trigger riots this time around. And he gave particular credit to Chief Constable Andy Marsh, suggesting that if he had been there in 1980 there wouldn’t have been riots. But the police have evolved in their training and tactics since then in part thanks to social scientists like Steve and his PhD students, now professors themselves in UK universities and often advising police on their responses to handling protests to avoid them turning into riots. The key point being to see crowds not as mad or bad but as highly minded and acting with reasons, and in contexts partly shaped by how the police themselves intervene. 3 Here’s a representative article urging shifts in the police’s construals of crowds at the time of the poll tax riots: https://onlinelibrary.wiley.com/doi/pdf/10.1002/(SICI)1099-0992(199807/08)28:4%3C509::AID-EJSP877%3E3.0.CO;2-C Social science imagination in action! I don’t know if Steve’s, his colleagues’ and others’ impact has been obliterated through incorporation, but I can see the link through time.

This is just one example. Play it out over the various domains I described earlier and you might see why I’m incredibly grateful to the social scientists present and past who through their work have shaped and framed my way of thinking and a stance toward the world which I believe would, in countless ways, be much poorer for its absence.

Ziyad Marar

Ziyad Marar is an author and president of global publishing at SAGE Publishing. His books include Judged: The Value of Being Misunderstood (Bloomsbury, 2018), Intimacy: Understanding the Subtle Power of Human Connection (Acumen Publishing, 2012), Deception (Acumen Publishing, 2008), and The Happiness Paradox (Reaktion Books 2003). He tweets @ZiyadMarar.

Why Social Science? Because It Can Help Contribute to AI That Benefits Society

Digital Scholarly Records are Facing New Risks

Civilisation – and Some Discontents

The Long Arm of Criminality

Exploring ‘Lost Person Behavior’ and the Science of Search and Rescue

What is the best strategy for finding someone missing in the wilderness? It’s complicated, but the method known as ‘Lost Person Behavior’ seems to offers some hope.

Survey Suggests University Researchers Feel Powerless to Take Climate Change Action

To feel able to contribute to climate action, researchers say they need to know what actions to take, how their institutions will support them and space in their workloads to do it.

Three Decades of Rural Health Research and a Bumper Crop of Insights from South Africa

A longitudinal research project project covering 31 villages in rural South Africa has led to groundbreaking research in many fields, including genomics, HIV/Aids, cardiovascular conditions and stroke, cognition and aging.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

This is interesting, and thought-provoking reading; I am reading it as part of the content for my MA in Creative Writing at Kingston School of Art. I am interested in your example of the ‘fundamental attribution error’ where we instantly ‘frame’ someone in a negative light in traffic, and your exposition on how this feeds through into many other examples. I think it is a shame though that you framed this as ‘imagined’, and that the imagination is often blamed for instances like this. As elucidated in A Critique of Pure Reason (Kant), you will note that it is the … Read more »

The most unkind remark made about the social sciences is that they are fuzzy science. Here in this article Ziyad Marar correctly explains that they are complex. And that they are infinitely worth pursueing. However, as Jordan Peterson and from a slightly different angle Douglas Murray might argue, there is a danger today that in this field the academic world has shifted so much to the left that University students are being misled in believing dismantelling statues for “righteous causes” is something brave and praiseworthy. I feel sure Mr. Marar would not be amongst those encouraging them, realising that different … Read more »

Webinar: How to Write and Structure an Article’s Front Matter

Webinar – what spurs action on climate change, webinar: how to get more involved with a journal and develop your career, science of team science 2024 virtual conference, webinar: how to collaborate across paradigms – embedding culture in mixed methods designs, american psychological association annual convention, societal impact of social sciences, humanities, and arts 2024, cspc 2024: 16th canadian science policy conference.

Imagine a graphic representation of your research's impact on public policy that you could share widely: Sage Policy Profiles

Customize your experience

Select your preferred categories.

Announcements
Business and Management INK

Communication

Higher education reform, open access, recent appointments, research ethics, international debate.

Academic Funding

Public Engagement

Recognition

Presentations

Science & social science, social science bites, the data bulletin.

New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

Thanks to a collaboration between the Inter-American Foundation (IAF) and the Social Science Research Council (SSRC), applications are now being accepted for […]

Social, Behavioral Scientists Eligible to Apply for NSF S-STEM Grants

Solicitations are now being sought for the National Science Foundation’s Scholarships in Science, Technology, Engineering, and Mathematics program, and in an unheralded […]

With COVID and Climate Change Showing Social Science’s Value, Why Cut it Now?

What are the three biggest challenges Australia faces in the next five to ten years? What role will the social sciences play in resolving these challenges? The Academy of the Social Sciences in Australia asked these questions in a discussion paper earlier this year. The backdrop to this review is cuts to social science disciplines around the country, with teaching taking priority over research.

Opportunity to Participate in RFI on Proposed National Secure Data Service

According to the United Nations Educational, Scientific, and Cultural Organization, scientific collaboration and diplomacy are key when trying to effectively address the […]

Public Interest Attorney Bryan Stevenson to Receive 2024 Moynihan Prize

Public interest attorney Bryan Stevenson, the founder and executive director of the Alabama-based Equal Justice Initiative, will receive the 2024 Daniel Patrick Moynihan Prize from the American Academy of Political and Social Science.

Young Explorers Award Honors Scholars at Nexus of Life and Social Science

Aiming to spur greater connections between the life and social sciences, Science magazine and NOMIS look to recognize young researchers through the NOMIS and Science Young Explorers Award.

New Podcast Series Applies Social Science to Social Justice Issues

Sage (the parent of Social Science Space) and the Surviving Society podcast have launched a collaborative podcast series, Social Science for Social […]

Big Think Podcast Series Launched by Canadian Federation of Humanities and Social Sciences

The Canadian Federation of Humanities and Social Sciences has launched the Big Thinking Podcast, a show series that features leading researchers in the humanities and social sciences in conversation about the most important and interesting issues of our time.

The We Society Explores Intersectionality and Single Motherhood

In a recently released episode of The We Society podcast, Ann Phoenix, a psychologist at University College London’s Institute of Education, spoke […]

Fourth Edition of ‘The Evidence’: Do Women Make Better Doctors?

In this issue of The Evidence newsletter, journalist Josephine Lethbridge examines why women doctors see better outcomes in their patients’ health.

Third Edition of ‘The Evidence’: How Can We Overcome Sexism in AI?

This month’s installment of The Evidence explores how leading ethics experts are responding to the urgent dilemma of gender bias in AI. […]

Second Edition of ‘The Evidence’ Examines Women and Climate Change

The second issue of The Evidence explores the intersection of gender inequality and the global climate crisis. Author Josephine Lethbridge recounts the […]

A Social Scientist Looks at the Irish Border and Its Future

‘What Do We Know and What Should We Do About the Irish Border?’ is a new book from Katy Hayward that applies social science to the existing issues and what they portend.

Brexit and the Decline of Academic Internationalism in the UK

Brexit seems likely to extend the hostility of the UK immigration system to scholars from European Union countries — unless a significant change of migration politics and prevalent public attitudes towards immigration politics took place in the UK. There are no indications that the latter will happen anytime soon.

Brexit and the Crisis of Academic Cosmopolitanism

A new report from the Royal Society about the effects on Brexit on science in the United Kingdom has our peripatetic Daniel Nehring mulling the changes that will occur in higher education and academic productivity.

Beyond Net-Zero Targets: When Do Companies Maximize Their Potential to Reduce Carbon Emissions?

Companies with a better understanding of climate change, the authors argue, have realized the need to plan actions beyond the business level.

How AI-Integration is Changing the Workplace

The authors describe how their study investigated how middle managers perceive the impacts of AI-system integration on their work characteristics in the financial services industry.

Keeping Qualitative Research Weird!

The authors urge qualitative researchers to retain what makes qualitative research different and powerful and yes… weird: the researcher’s voice, multitudes of potential data sources, and meaningful contextualization.

2024 Holberg Prize Goes to Political Theorist Achille Mbembe

Political theorist and public intellectual Achille Mbembe, among the most read and cited scholars from the African continent, has been awarded the 2024 Holberg Prize.

Edward Webster, 1942-2024: South Africa’s Pioneering Industrial Sociologist

Eddie Webster, sociologist and emeritus professor at the Southern Centre for Inequality Studies at the University of the Witwatersrand in South Africa, died on March 5, 2024, at age 82.

Charles V. Hamilton, 1929-2023: The Philosopher Behind ‘Black Power’

Political scientist Charles V. Hamilton, the tokenizer of the term ‘institutional racism,’ an apostle of the Black Power movement, and at times deemed both too radical and too deferential in how to fight for racial equity, died on November 18, 2023. He was 94.

National Academies Seeks Experts to Assess 2020 U.S. Census

The National Academies’ Committee on National Statistics seeks nominations for members of an ad hoc consensus study panel — sponsored by the U.S. Census Bureau — to review and evaluate the quality of the 2020 Census.

Will the 2020 Census Be the Last of Its Kind?

Could the 2020 iteration of the United States Census, the constitutionally mandated count of everyone present in the nation, be the last of its kind?

Will We See A More Private, But Less Useful, Census?

Census data can be pretty sensitive – it’s not just how many people live in a neighborhood, a town, a state or […]

Webinar – Trust in Science: Understanding the Trends and Implications for Science Communication

Recent survey data show declines in trust in science that mirror earlier trends for other institutions, including journalism and government. New research […]

Celebrating 20 Years of an Afrocentric Small Scholarly Press

To mark the Black- and female-owned Universal Write Publications’ 20th anniversary, Sage’s Geane De Lima asked UWP fonder Ayo Sekai some questions about UWP’s past, present and future.

Free Online Course Reveals The Art of ChatGPT Interactions

You’ve likely heard the hype around artificial intelligence, or AI, but do you find ChatGPT genuinely useful in your professional life? A free course offered by Sage Campus could change all th

How Social Science Can Hurt Those It Loves

David Canter rues the way psychologists and other social scientists too often emasculate important questions by forcing them into the straitjacket of limited scientific methods.

The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

Based on a study of how research is cited in national and local media sources, Andy Tattersall shows how research is often poorly represented in the media and suggests better community standards around linking to original research could improve trust in mainstream media.

Research Integrity Should Not Mean Its Weaponization

Commenting on the trend for the politically motivated forensic scrutiny of the research records of academics, Till Bruckner argues that singling out individuals in this way has a chilling effect on academic freedom and distracts from efforts to address more important systemic issues in research integrity.

Webinar – What Spurs Action on Climate Change?

Policies to combat rapid climate change have been met with resistance. This webinar will investigate the psychological factors inhibiting actions and policy […]

Spring 2024 Social, Behavioral and Economic Sciences Advisory Committee Meeting

The advisory committee for the U.S. National Science Foundation’s Social, Behavioral and Economic Sciences Directorate meets twice yearly to provide advice, recommendations […]

Webinar – Navigating the Era of Artificial Intelligence: Achieving Human-AI Harmony

This two-part webinar series, funded through the Hauser Policy Impact Fund, will explore the Division of Behavioral and Social Sciences and Education’s […]

New Opportunity to Support Government Evaluation of Public Participation and Community Engagement Now Open

The President’s Management Agenda Learning Agenda: Public Participation & Community Engagement Evidence Challenge is dedicated to forming a strategic, evidence-based plan that federal agencies and external researchers can use to solve big problems.

Universities Should Reimagine Governance Along Co-Operative Lines

Instead of adhering to a corporate model based on individual achievement, the authors argue that universities need to shift towards co-operative governance that fosters collaborative approaches to teaching and research

Striving for Linguistic Diversity in Scientific Research

Each country has its own unique role to play in promoting greater linguistic diversity in scientific communication.

The Power of Fuzzy Expectations: Enhancing Equity in Australian Higher Education

Having experienced firsthand the transformational power of education, the authors wanted to shed light on the contemporary challenges faced by regional and remote university students.

Social sciences can also inform the design and creation of ethical frameworks and guidelines for AI development and for deployment into systems. Social scientists can contribute expertise: on data quality, equity, and reliability; on how bias manifests in AI algorithms and decision-making processes; on how AI technologies impact marginalized communities and exacerbate existing inequities; and on topics such as fairness, transparency, privacy, and accountability.

Drawing on a study of Crossref DOI data, Martin Eve finds evidence to suggest that the current standard of digital preservation could fall worryingly short of ensuring persistent accurate record of scholarly works.

Infrastructure

To Better Forecast AI, We Need to Learn Where Its Money Is Pointing

By carefully interrogating the system of economic incentives underlying innovations and how technologies are monetized in practice, we can generate a better understanding of the risks, both economic and technological, nurtured by a market’s structure.

Why Social Science? Because It Makes an Outsized Impact on Policy

Euan Adie, founder of Altmetric and Overton and currently Overton’s managing director, answers questions about the outsized impact that SBS makes on policy and his work creating tools to connect the scholarly and policy worlds.

Maybe You Can’t Buy Happinesss, But You Can Teach About It

When you deliver a university course that makes students happier, everybody wants to know what the secret is. What are your tips? […]

There’s Something in the Air, Part 2 – But It’s Not a Miasma

Robert Dingwall looks at the once dominant role that miasmatic theory had in public health interventions and public policy.

The Fog of War

David Canter considers the psychological and organizational challenges to making military decisions in a war.

The TV series Civilisation shows us many beautiful images and links them with a compelling narrative. But it is a narrative of its time and place.

Philip Rubin: FABBS’ Accidental Essential Man Linking Research and Policy

As he stands down from a two-year stint as the president of the Federation of Associations in Behavioral & Brain Sciences, or FABBS, Social Science Space took the opportunity to download a fraction of the experiences of cognitive psychologist Philip Rubin, especially his experiences connecting science and policy.

David Canter considers the daily reminders of details of our actions that have been caused by criminality.

Why Don’t Algorithms Agree With Each Other?

David Canter reviews his experience of filling in automated forms online for the same thing but getting very different answers, revealing the value systems built into these supposedly neutral processes.

A Black History Addendum to the American Music Industry

The new editor of the case study series on the music industry discusses the history of Black Americans in the recording industry.

A Behavioral Scientist’s Take on the Dangers of Self-Censorship in Science

The word censorship might bring to mind authoritarian regimes, book-banning, and restrictions on a free press, but Cory Clark, a behavioral scientist at […]

Jonathan Breckon On Knowledge Brokerage and Influencing Policy

Overton spoke with Jonathan Breckon to learn about knowledge brokerage, influencing policy and the potential for technology and data to streamline the research-policy interface.

Research for Social Good Means Addressing Scientific Misconduct

Social Science Space’s sister site, Methods Space, explored the broad topic of Social Good this past October, with guest Interviewee Dr. Benson Hong. Here Janet Salmons and him talk about the Academy of Management Perspectives journal article.

NSF Looks Headed for a Half-Billion Dollar Haircut

Funding for the U.S. National Science Foundation would fall by a half billion dollars in this fiscal year if a proposed budget the House of Representatives’ Appropriations Committee takes effect – the first cut to the agency’s budget in several years.

NSF Responsible Tech Initiative Looking at AI, Biotech and Climate

The U.S. National Science Foundation’s new Responsible Design, Development, and Deployment of Technologies (ReDDDoT) program supports research, implementation, and educational projects for multidisciplinary, multi-sector teams

Digital Transformation Needs Organizational Talent and Leadership Skills to Be Successful

Who drives digital change – the people of the technology? Katharina Gilli explains how her co-authors worked to address that question.

Six Principles for Scientists Seeking Hiring, Promotion, and Tenure

The negative consequences of relying too heavily on metrics to assess research quality are well known, potentially fostering practices harmful to scientific research such as p-hacking, salami science, or selective reporting. To address this systemic problem, Florian Naudet, and collegues present six principles for assessing scientists for hiring, promotion, and tenure.

Book Review: The Oxford Handbook of Creative Industries

Candace Jones, Mark Lorenzen, Jonathan Sapsed , eds.: The Oxford Handbook of Creative Industries. Oxford: Oxford University Press, 2015. 576 pp. $170.00, […]

Biden Administration Releases ‘Blueprint’ For Using Social and Behavioral Science in Policy

U.S. President Joseph Biden’s administration has laid down a marker buttressing the use of social and behavioral science in crafting policies for the federal government by releasing a 102-page Blueprint for the Use of Social and Behavioral Science to Advance Evidence-Based Policymaking.

Daniel Kahneman, 1934-2024: The Grandfather of Behavioral Economics

Nobel laureate Daniel Kahneman, whose psychological insights in both the academic and the public spheres revolutionized how we approach economics, has died […]

Canadian Librarians Suggest Secondary Publishing Rights to Improve Public Access to Research

The Canadian Federation of Library Associations recently proposed providing secondary publishing rights to academic authors in Canada.

Webinar: How Can Public Access Advance Equity and Learning?

The U.S. National Science Foundation and the American Association for the Advancement of Science have teamed up present a 90-minute online session examining how to balance public access to federally funded research results with an equitable publishing environment.

Open Access in the Humanities and Social Sciences in Canada: A Conversation

Five organizations representing knowledge networks, research libraries, and publishing platforms joined the Federation of Humanities and Social Sciences to review the present and the future of open access — in policy and in practice – in Canada

A Former Student Reflects on How Daniel Kahneman Changed Our Understanding of Human Nature

Daniel Read argues that one way the late Daniel Kahneman stood apart from other researchers is that his work was driven by a desire not merely to contribute to a research field, but to create new fields.

Four Reasons to Stop Using the Word ‘Populism’

Beyond poor academic practice, the careless use of the word ‘populism’ has also had a deleterious impact on wider public discourse, the authors argue.

The Added Value of Latinx and Black Teachers

As the U.S. Congress debates the reauthorization of the Higher Education Act, a new paper in Policy Insights from the Behavioral and Brain Sciences urges lawmakers to focus on provisions aimed at increasing the numbers of black and Latinx teachers.

A Collection: Behavioral Science Insights on Addressing COVID’s Collateral Effects

To help in decisions surrounding the effects and aftermath of the COVID-19 pandemic, the the journal ‘Policy Insights from the Behavioral and Brain Sciences’ offers this collection of articles as a free resource.

Susan Fiske Connects Policy and Research in Print

Psychologist Susan Fiske was the founding editor of the journal Policy Insights from the Behavioral and Brain Sciences. In trying to reach a lay audience with research findings that matter, she counsels stepping a bit outside your academic comfort zone.

Mixed Methods As A Tool To Research Self-Reported Outcomes From Diverse Treatments Among People With Multiple Sclerosis

What does heritage mean to you?

Personal Information Management Strategies in Higher Education

Working Alongside Artificial Intelligence Key Focus at Critical Thinking Bootcamp 2022

SAGE Publishing — the parent of Social Science Space – will hold its Third Annual Critical Thinking Bootcamp on August 9. Leaning more and register here

Watch the Forum: A Turning Point for International Climate Policy

On May 13, the American Academy of Political and Social Science hosted an online seminar, co-sponsored by SAGE Publishing, that featured presentations […]

Event: Living, Working, Dying: Demographic Insights into COVID-19

On Friday, April 23rd, join the Population Association of America and the Association of Population Centers for a virtual congressional briefing. The […]

Connecting Legislators and Researchers, Leads to Policies Based on Scientific Evidence

The author’s team is developing ways to connect policymakers with university-based researchers – and studying what happens when these academics become the trusted sources, rather than those with special interests who stand to gain financially from various initiatives.

Public Policy

Rob Ford on Immigration

Opinions on immigration are not set in stone, suggests Rob Ford – but they may be set in generations. Zeroing in on the experience of the United Kingdom since the end of World War II, Ford – a political scientist at the University of Manchester – explains how this generation’s ‘other’ becomes the next generation’s ‘neighbor.’

Economist Kaye Husbands Fealing to Lead NSF’s Social Science Directorate

Kaye Husbands Fealing, an economist who has done pioneering work in the “science of broadening participation,” has been named the new leader of the U.S. National Science Foundation’s Directorate for Social, Behavioral and Economic Sciences.

Jane M. Simoni Named New Head of OBSSR

Clinical psychologist Jane M. Simoni has been named to head the U.S. National Institutes of Health’s Office of Behavioral and Social Sciences Research

Canada’s Federation For Humanities and Social Sciences Welcomes New Board Members

Annie Pilote, dean of the faculty of graduate and postdoctoral studies at the Université Laval, was named chair of the Federation for the Humanities and Social Sciences at its 2023 virtual annual meeting last month. Members also elected Debra Thompson as a new director on the board.

AAPSS Names Eight as 2024 Fellows

The American Academy of Political and Social Science today named seven scholars and one journalist as its 2024 fellows class.

National Academies Looks at How to Reduce Racial Inequality In Criminal Justice System

To address racial and ethnic inequalities in the U.S. criminal justice system, the National Academies of Sciences, Engineering and Medicine just released “Reducing Racial Inequality in Crime and Justice: Science, Practice and Policy.”

Survey Examines Global Status Of Political Science Profession

The ECPR-IPSA World of Political Science Survey 2023 assesses political science scholar’s viewpoints on the global status of the discipline and the challenges it faces, specifically targeting the phenomena of cancel culture, self-censorship and threats to academic freedom of expression.

Report: Latest Academic Freedom Index Sees Global Declines

The latest update of the global Academic Freedom Index finds improvements in only five countries

Analyzing the Impact: Social Media and Mental Health

The social and behavioral sciences supply evidence-based research that enables us to make sense of the shifting online landscape pertaining to mental health. We’ll explore three freely accessible articles (listed below) that give us a fuller picture on how TikTok, Instagram, Snapchat, and online forums affect mental health.

The Risks Of Using Research-Based Evidence In Policymaking

With research-based evidence increasingly being seen in policy, we should acknowledge that there are risks that the research or ‘evidence’ used isn’t suitable or can be accidentally misused for a variety of reasons.

Surveys Provide Insight Into Three Factors That Encourage Open Data and Science

Over a 10-year period Carol Tenopir of DataONE and her team conducted a global survey of scientists, managers and government workers involved in broad environmental science activities about their willingness to share data and their opinion of the resources available to do so (Tenopir et al., 2011, 2015, 2018, 2020). Comparing the responses over that time shows a general increase in the willingness to share data (and thus engage in Open Science).

Maintaining Anonymity In Double-Blind Peer Review During The Age of Artificial Intelligence

The double-blind review process, adopted by many publishers and funding agencies, plays a vital role in maintaining fairness and unbiasedness by concealing the identities of authors and reviewers. However, in the era of artificial intelligence (AI) and big data, a pressing question arises: can an author’s identity be deduced even from an anonymized paper (in cases where the authors do not advertise their submitted article on social media)?

Hype Terms In Research: Words Exaggerating Results Undermine Findings

The claim that academics hype their research is not news. The use of subjective or emotive words that glamorize, publicize, embellish or exaggerate results and promote the merits of studies has been noted for some time and has drawn criticism from researchers themselves. Some argue hyping practices have reached a level where objectivity has been replaced by sensationalism and manufactured excitement. By exaggerating the importance of findings, writers are seen to undermine the impartiality of science, fuel skepticism and alienate readers.

Five Steps to Protect – and to Hear – Research Participants

Jasper Knight identifies five key issues that underlie working with human subjects in research and which transcend institutional or disciplinary differences.

New Tool Promotes Responsible Hiring, Promotion, and Tenure in Research Institutions

Modern-day approaches to understanding the quality of research and the careers of researchers are often outdated and filled with inequalities. These approaches […]

There’s Something In the Air…But Is It a Virus? Part 1

The historic Hippocrates has become an iconic figure in the creation myths of medicine. What can the body of thought attributed to him tell us about modern responses to COVID?

Tavneet Suri on Universal Basic Income

Economist Tavneet Suri discusses fieldwork she’s done in handing our cash directly to Kenyans in poor and rural parts of Kenya, and what the generally good news from that work may herald more broadly.

Alex Edmans on Confirmation Bias

In this Social Science Bites podcast, Edmans, a professor of finance at London Business School and author of the just-released “May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – And What We Can Do About It,” reviews the persistence of confirmation bias even among professors of finance.

Tejendra Pherali on Education and Conflict

Tejendra Pherali, a professor of education, conflict and peace at University College London, researches the intersection of education and conflict around the world.

Gamification as an Effective Instructional Strategy

Gamification—the use of video game elements such as achievements, badges, ranking boards, avatars, adventures, and customized goals in non-game contexts—is certainly not a new thing.

Harnessing the Tide, Not Stemming It: AI, HE and Academic Publishing

Who will use AI-assisted writing tools — and what will they use them for? The short answer, says Katie Metzler, is everyone and for almost every task that involves typing.

Immigration Court’s Active Backlog Surpasses One Million

In the first post from a series of bulletins on public data that social and behavioral scientists might be interested in, Gary Price links to an analysis from the Transactional Records Access Clearinghouse.

Webinar Discusses Promoting Your Article

The next in SAGE Publishing’s How to Get Published webinar series focuses on promoting your writing after publication. The free webinar is set for November 16 at 4 p.m. BT/11 a.m. ET/8 a.m. PT.

Webinar Examines Open Access and Author Rights

The next in SAGE Publishing’s How to Get Published webinar series honors International Open Access Week (October 24-30). The free webinar is […]

Ping, Read, Reply, Repeat: Research-Based Tips About Breaking Bad Email Habits

At a time when there are so many concerns being raised about always-on work cultures and our right to disconnect, email is the bane of many of our working lives.

New Dataset Collects Instances of ‘Contentious Politics’ Around the World

The European Research Center is funding the Global Contentious Politics Dataset, or GLOCON, a state-of-the-art automated database curating information on political events — including confrontations, political turbulence, strikes, rallies, and protests

Matchmaking Research to Policy: Introducing Britain’s Areas of Research Interest Database

Kathryn Oliver discusses the recent launch of the United Kingdom’s Areas of Research Interest Database. A new tool that promises to provide a mechanism to link researchers, funders and policymakers more effectively collaboratively and transparently.

Watch The Lecture: The ‘E’ In Science Stands For Equity

According to the National Science Foundation, the percentage of American adults with a great deal of trust in the scientific community dropped […]

Watch a Social Scientist Reflect on the Russian Invasion of Ukraine

“It’s very hard,” explains Sir Lawrence Freedman, “to motivate people when they’re going backwards.”

Dispatches from Social and Behavioral Scientists on COVID

Has the ongoing COVID-19 pandemic impacted how social and behavioral scientists view and conduct research? If so, how exactly? And what are […]

Contemporary Politics Focus of March Webinar Series

This March, the Sage Politics team launches its first Politics Webinar Week. These webinars are free to access and will be delivered by contemporary politics experts —drawn from Sage’s team of authors and editors— who range from practitioners to instructors.

New Thought Leadership Webinar Series Opens with Regional Looks at Research Impact

Research impact will be the focus of a new webinar series from Epigeum, which provides online courses for universities and colleges. The […]

Impact metrics
Early Career
In Memorium
Curated-Collection Page Links
Science communication
True Crime: Insight Into The Human Fascination With The Who-Done-It
Melissa Kearney on Marriage and Children
Raffaella Sadun on Effective Management
Norman Denzin, 1941-2023: The Father of Qualitative Research

Subscribe to our mailing list

Get the latest news from the social and behavioral science community delivered straight to your inbox.

Thanks! Please check your inbox or spam folder to confirm your subscription.

For Librarians
For Publishers
For Researchers
OAPEN Home

The Public Value of the Social Sciences

An Interpretive Essay

Publisher website

Publication date and place, classification.

Imported or submitted locally

Export search results

The export option will allow you to export the current search results of the entered query to a file. Differen formats are available for download. To export the items, click on the button corresponding with the preferred download format.

A logged-in user can export up to 15000 items. If you're not logged in, you can export no more than 500 items.

To select a subset of the search results, click "Selective Export" button and make a selection of the items you want to export. The amount of items that can be exported at once is similarly restricted as the full export.

After making a selection, click one of the export format buttons. The amount of items that will be exported is indicated in the bubble next to export format.

Information

Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

Active Journals
Find a Journal
Proceedings Series
For Authors
For Reviewers
For Editors
For Librarians
For Publishers
For Societies
For Conference Organizers
Open Access Policy
Institutional Open Access Program
Special Issues Guidelines
Editorial Process
Research and Publication Ethics
Article Processing Charges
Testimonials
Preprints.org
SciProfiles
Encyclopedia

From Intersex Activism to Law-Making—The Legal Ban of Intersex Genital Mutilation (IGM) in Greece

Journal Description

Social sciences.

Open Access — free for readers, with article processing charges (APC) paid by authors or their institutions.
High visibility: indexed within Scopus , ESCI (Web of Science) , RePEc , and other databases .
Journal Rank: CiteScore - Q2 ( General Social Sciences )
Rapid Publication: manuscripts are peer-reviewed and a first decision is provided to authors approximately 27.7 days after submission; acceptance to publication is undertaken in 5.3 days (median values for papers published in this journal in the second half of 2023).
Recognition of Reviewers: reviewers who provide timely, thorough peer-review reports receive vouchers entitling them to a discount on the APC of their next publication in any MDPI journal, in appreciation of the work done.

Latest Articles

Journal Menu

Social Sciences Home
Aims & Scope
Editorial Board
Reviewer Board
Topical Advisory Panel
Instructions for Authors

Special Issues

Sections & Collections
Article Processing Charge
Indexing & Archiving
Editor’s Choice Articles
Most Cited & Viewed
Journal Statistics
Journal History
Journal Awards
Editorial Office

Journal Browser

arrow_forward_ios Forthcoming issue arrow_forward_ios Current issue
Vol. 13 (2024)
Vol. 12 (2023)
Vol. 11 (2022)
Vol. 10 (2021)
Vol. 9 (2020)
Vol. 8 (2019)
Vol. 7 (2018)
Vol. 6 (2017)
Vol. 5 (2016)
Vol. 4 (2015)
Vol. 3 (2014)
Vol. 2 (2013)
Vol. 1 (2012)

Highly Accessed Articles

Latest books, e-mail alert.

Conferences

Topical collections, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

APPLY NOW --> REQUEST INFO

Apply Today

Ready to apply to Penn LPS Online? Apply Now

Learn more about Penn LPS Online

Request More Information

Humanities vs. social science: exploring the dichotomy

Photo representation of a group of people

The difference between humanities and social science isn’t always immediately apparent. Although both disciplines focus on the human experience, they each view it from a unique lens. This article provides an overview of the different approaches, areas of study, and career paths for your humanities vs. social science consideration so you can pinpoint which option best aligns with your goals.

What are the humanities?

Through the critical study of literature, art, language, history, and philosophy, humanities allow us to develop a deeper understanding of the human condition and how it is documented.

The origin of the modern idea of the humanities can be traced back to the mid-5th century BCE, with the Classical Greek “paideia,” a course of general education, as well as in Cicero’s “humanitas” (which means human nature).

With a focus on the power of human expression and shared experiences, the humanities provide an opportunity to cultivate empathy and create dialogue between people of different beliefs and perspectives.

Unlike the social sciences which focus on observable patterns, humanities focus on abstract or theoretical ideas when examining everything from ethics and poetry to how to live a life of meaning. Students of the humanities primarily employ a qualitative research approach, making analytical, moral or speculative interpretations regarding the traditions, creations, and cultures of the past and the present.

By studying forms of art, literary works, historical events, philosophical ideas, and spiritual practices, practitioners of the humanities seek to better understand the individual within the broader context of human culture and society.

Areas of study

The humanities encompass many engaging fields within a comprehensive liberal arts education. For example, philosophy involves the logical, abstract, and methodical consideration of fundamental truths such as the nature of reality, knowledge, and human existence.

The study of history focuses on the cultural, social, political, economic, religious, and other changes in society over time, while art history and theory courses may take a critical eye to everything from painting, drawing, and sculpture to photography, filmmaking, or music.

Literature studies, such as ENLT 1200: Writing about Literature and Culture at Penn LPS Online, provide an overview of foundational skills and strategies to write clear, concise, and persuasive critical analyses of different form of literary texts.

Other examples of common areas of study within the humanities include the classics, drama and music, gender studies, regional studies, ethnic studies, and religious studies.

Career paths in humanities

The critical thinking, communication, and interpersonal skills that you develop by earning a humanities degree may be applied to a wide variety of career paths, depending on your preferences and goals.

If you’re interested in teaching at the pre-school, elementary, or high school level, you’ll have a chance to put your skills in communication, empathy, and resourcefulness to good use. Keep in mind that you will also need to obtain certification or licensure to practice in addition to your bachelor degree.

Communication, media, and publishing

In the fields of communications and publishing, roles such as copywriter, editor, author, journalist, technical writer, public relations professional, or communications specialist require many of the competencies developed while studying the humanities, including creativity, research abilities, writing, and critical thinking.

History and the arts

If you’re passionate about history or the arts, you may want to pursue a career as an archivist, curator, conservator, or museum technician. These professions all require sound organizational, research, and analytical skills, which are critical in determining the origin, history, and relevance of records, documents and artwork. Some employers may require that you earn a master’s degree to be considered for these roles.

What are social sciences?

Social science is the study of how humans behave and interact within societies, and key branches include anthropology, political science, psychology, sociology, and economics. It began to emerge as a distinct field in the early 19th century, making it a relatively new discipline compared to the humanities.

Social scientists investigate government, the economy, family structures, and social institutions to gain insight into what drives human behavior and what conditions allow people to flourish. Not only do the findings of social science inform education programs, public policy, and urban planning, they also help spearhead strategies to create more inclusive and effective institutions within society.

Unlike the natural sciences, which study the physical world through fields such as chemistry, biology, and physics, social science examines the relationships and cultures within the constructed world of human beings.

Social science frequently employs a scientific approach of quantitative data analysis to study different facets of society. For example, sociologists research and collect data on crime and poverty, analyze it, and draw conclusions that are used to guide public policy.

However, social scientists may also use qualitative approaches (such as interviews or focus groups) to increase their understanding regarding the motivations and interactions of individuals and groups.

Two fields most commonly associated with social science are sociology and psychology. Sociology is the scientific study of patterns of human social behavior, relationships, and interactions, including how they are shaped by social institutions. Psychology explores how people’s emotions, thoughts, and behaviors are affected by actual or imagined interactions with others.

Anthropology is a broad discipline that delves into the history and development of human societies and cultures, including their languages, belief systems, material goods, and social structures.

Economics is concerned with the production, distribution, and consumption of goods and services with a focus on decision-making by individuals, governments, and businesses.

Political science examines topics such as political theory, ideologies, and international relations to help understand how local, national, and international governments function and impact societies.

There is some disagreement as to whether history, geography, law, and linguistics should be classified as part of the humanities or social sciences—and consequently viewed from a theoretical or scientific perspective.

Career paths in social science

The analytical, research, and problem-solving skills obtained while pursuing a degree in social science can be applied in a wide range of professional paths, including the role of a social scientist.

Social scientists use the scientific method to study how particular aspects of human society and relationships function, change, and coexist. Examples of different types of social scientists include economists, political scientists, sociologists, psychologists, historians, and geographers.

The entry-level education required to become a social scientist is typically a master’s or doctoral degree. Below are other professional opportunities to consider if you earn a bachelor’s in social science.

Counseling, therapy, and social services

If you’re interested in helping others prevent and cope with challenges in their everyday lives, in-demand roles that you may want to pursue include social worker, social and community service manager, and substance abuse, behavioral disorder, or mental health counselor.

Job growth for these rewarding career paths is expected to increase by 9% , 12% , and 22% , respectively, by 2031, all of which are faster than average.

The combination of strong analytical and research skills with knowledge of what drives human behavior can be applied to multiple facets of business, including data analysis, marketing, communications, management, and human resources.

For example, market research analysts help companies understand consumer buying preferences and demographics, while business intelligence analysts create financial and market data reports. Demand for these roles is expected to increase 19% and 11% , respectively, by 2031.

Government and public administration

Robust communication, interpretive, and critical thinking skills—and expertise in political science and sociology—may open opportunities in government or civil services positions, including policy analyst, public affairs specialist, campaign manager, or communications officer.

If you earn an undergraduate degree focused on political science, psychology, history, or economics, you may want to consider furthering your education by earning a Juris Doctor (JD) degree from an accredited law school, taking the bar exam, and pursuing a career as a lawyer.

Why study the humanities or social sciences?

A degree in the humanities is generally less career-focused or specialized than in social sciences because you’ll explore topics from a historical or theoretical perspective, rather than a practical one.

However, whichever discipline you choose, you’ll have the opportunity to develop valuable skills that can be applied in your personal and professional life.

Analytical abilities: Obtain strong analytical skills including logical reasoning, problem-solving, and decision-making. And learn how to reach logical conclusions through both qualitative and quantitative analyses of numerical data, survey results, and written sources.

Creativity and empathy: Learn how to come up with original ideas and solutions to problems, while obtaining an understanding and appreciation for the feelings and viewpoints of others.

Critical thinking: Develop your ability to make rational connections between ideas, recognize and develop arguments, find solutions to problems, and consider how your own assumptions may affect your interpretations.

Written and verbal communication: Learn how to clearly and concisely author persuasive essays and research papers and deliver compelling evidence-based arguments in discussions and presentations.

Research: Discover how to identify and analyze reputable sources, extract important evidence and data, and present sound conclusions based on qualitative and quantitative research.

Ready to enhance your professional skill set?

With lessons from expert practitioners and strategies for self-knowledge, the concentration in Leadership and Communication for the Bachelor of Applied Arts and Sciences (BAAS) program at Penn LPS Online will help you unlock your leadership potential.

This interdisciplinary Ivy League degree provides the practical knowledge and critical thinking skills needed to develop career-advancing leadership and communication skills. Tailor your program to your personal strengths and professional interests as you combine analytical and reflective studies with practical introductions to data analysis and strategies in global leadership.

The BAAS in Leadership and Communication also prepares you to:

Apply leadership lessons from the humanities and social sciences
Discover how to integrate positive psychology into your leadership style
Explore competing perspectives on leadership ethics
Learn how to use quantitative data in decision-making
Develop strategies and skills to persuade via verbal, visual, and written communication
Understand and apply research insights in economics, psychology, and other social sciences

If you haven’t already, apply to Penn LPS Online today and register for the Leadership and Communication concentration for the B AAS degree , or explore our Certificate in Leadership and Communication . You can also view our course guide to learn more about what’s available in any upcoming term.

https://www.bestcolleges.com/humanities/what-is-humanities/ https://www.coursera.org/articles/what-is-a-humanities-major https://www.investopedia.com/terms/s/social-science.asp https://www.bestcolleges.com/blog/what-is-social-science/

Reference Examples

More than 100 reference examples and their corresponding in-text citations are presented in the seventh edition Publication Manual . Examples of the most common works that writers cite are provided on this page; additional examples are available in the Publication Manual .

To find the reference example you need, first select a category (e.g., periodicals) and then choose the appropriate type of work (e.g., journal article ) and follow the relevant example.

When selecting a category, use the webpages and websites category only when a work does not fit better within another category. For example, a report from a government website would use the reports category, whereas a page on a government website that is not a report or other work would use the webpages and websites category.

Also note that print and electronic references are largely the same. For example, to cite both print books and ebooks, use the books and reference works category and then choose the appropriate type of work (i.e., book ) and follow the relevant example (e.g., whole authored book ).

Examples on these pages illustrate the details of reference formats. We make every attempt to show examples that are in keeping with APA Style’s guiding principles of inclusivity and bias-free language. These examples are presented out of context only to demonstrate formatting issues (e.g., which elements to italicize, where punctuation is needed, placement of parentheses). References, including these examples, are not inherently endorsements for the ideas or content of the works themselves. An author may cite a work to support a statement or an idea, to critique that work, or for many other reasons. For more examples, see our sample papers .

Reference examples are covered in the seventh edition APA Style manuals in the Publication Manual Chapter 10 and the Concise Guide Chapter 10

Related handouts

Common Reference Examples Guide (PDF, 147KB)
Reference Quick Guide (PDF, 225KB)

Textual Works

Textual works are covered in Sections 10.1–10.8 of the Publication Manual . The most common categories and examples are presented here. For the reviews of other works category, see Section 10.7.

Journal Article References
Magazine Article References
Newspaper Article References
Blog Post and Blog Comment References
UpToDate Article References
Book/Ebook References
Diagnostic Manual References
Children’s Book or Other Illustrated Book References
Classroom Course Pack Material References
Religious Work References
Chapter in an Edited Book/Ebook References
Dictionary Entry References
Wikipedia Entry References
Report by a Government Agency References
Report with Individual Authors References
Brochure References
Ethics Code References
Fact Sheet References
ISO Standard References
Press Release References
White Paper References
Conference Presentation References
Conference Proceeding References
Published Dissertation or Thesis References
Unpublished Dissertation or Thesis References
ERIC Database References
Preprint Article References

Data and Assessments

Data sets are covered in Section 10.9 of the Publication Manual . For the software and tests categories, see Sections 10.10 and 10.11.

Data Set References
Toolbox References

Audiovisual Media

Audiovisual media are covered in Sections 10.12–10.14 of the Publication Manual . The most common examples are presented together here. In the manual, these examples and more are separated into categories for audiovisual, audio, and visual media.

Artwork References
Clip Art or Stock Image References
Film and Television References
Musical Score References
Online Course or MOOC References
Podcast References
PowerPoint Slide or Lecture Note References
Radio Broadcast References
TED Talk References
Transcript of an Audiovisual Work References
YouTube Video References

Online Media

Online media are covered in Sections 10.15 and 10.16 of the Publication Manual . Please note that blog posts are part of the periodicals category.

Facebook References
Instagram References
LinkedIn References
Online Forum (e.g., Reddit) References
TikTok References
X References
Webpage on a Website References
Clinical Practice References
Open Educational Resource References
Whole Website References

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 03 June 2024

Applying large language models for automated essay scoring for non-native Japanese

Wenchao Li 1 &
Haitao Liu 2

Humanities and Social Sciences Communications volume 11 , Article number: 723 ( 2024 ) Cite this article

254 Accesses

2 Altmetric

Metrics details

Language and linguistics

Recent advancements in artificial intelligence (AI) have led to an increased use of large language models (LLMs) for language assessment tasks such as automated essay scoring (AES), automated listening tests, and automated oral proficiency assessments. The application of LLMs for AES in the context of non-native Japanese, however, remains limited. This study explores the potential of LLM-based AES by comparing the efficiency of different models, i.e. two conventional machine training technology-based methods (Jess and JWriter), two LLMs (GPT and BERT), and one Japanese local LLM (Open-Calm large model). To conduct the evaluation, a dataset consisting of 1400 story-writing scripts authored by learners with 12 different first languages was used. Statistical analysis revealed that GPT-4 outperforms Jess and JWriter, BERT, and the Japanese language-specific trained Open-Calm large model in terms of annotation accuracy and predicting learning levels. Furthermore, by comparing 18 different models that utilize various prompts, the study emphasized the significance of prompts in achieving accurate and reliable evaluations using LLMs.

Accurate structure prediction of biomolecular interactions with AlphaFold 3

Scaling neural machine translation to 200 languages

Highly accurate protein structure prediction with AlphaFold

Conventional machine learning technology in aes.

AES has experienced significant growth with the advancement of machine learning technologies in recent decades. In the earlier stages of AES development, conventional machine learning-based approaches were commonly used. These approaches involved the following procedures: a) feeding the machine with a dataset. In this step, a dataset of essays is provided to the machine learning system. The dataset serves as the basis for training the model and establishing patterns and correlations between linguistic features and human ratings. b) the machine learning model is trained using linguistic features that best represent human ratings and can effectively discriminate learners’ writing proficiency. These features include lexical richness (Lu, 2012 ; Kyle and Crossley, 2015 ; Kyle et al. 2021 ), syntactic complexity (Lu, 2010 ; Liu, 2008 ), text cohesion (Crossley and McNamara, 2016 ), and among others. Conventional machine learning approaches in AES require human intervention, such as manual correction and annotation of essays. This human involvement was necessary to create a labeled dataset for training the model. Several AES systems have been developed using conventional machine learning technologies. These include the Intelligent Essay Assessor (Landauer et al. 2003 ), the e-rater engine by Educational Testing Service (Attali and Burstein, 2006 ; Burstein, 2003 ), MyAccess with the InterlliMetric scoring engine by Vantage Learning (Elliot, 2003 ), and the Bayesian Essay Test Scoring system (Rudner and Liang, 2002 ). These systems have played a significant role in automating the essay scoring process and providing quick and consistent feedback to learners. However, as touched upon earlier, conventional machine learning approaches rely on predetermined linguistic features and often require manual intervention, making them less flexible and potentially limiting their generalizability to different contexts.

In the context of the Japanese language, conventional machine learning-incorporated AES tools include Jess (Ishioka and Kameda, 2006 ) and JWriter (Lee and Hasebe, 2017 ). Jess assesses essays by deducting points from the perfect score, utilizing the Mainichi Daily News newspaper as a database. The evaluation criteria employed by Jess encompass various aspects, such as rhetorical elements (e.g., reading comprehension, vocabulary diversity, percentage of complex words, and percentage of passive sentences), organizational structures (e.g., forward and reverse connection structures), and content analysis (e.g., latent semantic indexing). JWriter employs linear regression analysis to assign weights to various measurement indices, such as average sentence length and total number of characters. These weights are then combined to derive the overall score. A pilot study involving the Jess model was conducted on 1320 essays at different proficiency levels, including primary, intermediate, and advanced. However, the results indicated that the Jess model failed to significantly distinguish between these essay levels. Out of the 16 measures used, four measures, namely median sentence length, median clause length, median number of phrases, and maximum number of phrases, did not show statistically significant differences between the levels. Additionally, two measures exhibited between-level differences but lacked linear progression: the number of attributives declined words and the Kanji/kana ratio. On the other hand, the remaining measures, including maximum sentence length, maximum clause length, number of attributive conjugated words, maximum number of consecutive infinitive forms, maximum number of conjunctive-particle clauses, k characteristic value, percentage of big words, and percentage of passive sentences, demonstrated statistically significant between-level differences and displayed linear progression.

Both Jess and JWriter exhibit notable limitations, including the manual selection of feature parameters and weights, which can introduce biases into the scoring process. The reliance on human annotators to label non-native language essays also introduces potential noise and variability in the scoring. Furthermore, an important concern is the possibility of system manipulation and cheating by learners who are aware of the regression equation utilized by the models (Hirao et al. 2020 ). These limitations emphasize the need for further advancements in AES systems to address these challenges.

Deep learning technology in AES

Deep learning has emerged as one of the approaches for improving the accuracy and effectiveness of AES. Deep learning-based AES methods utilize artificial neural networks that mimic the human brain’s functioning through layered algorithms and computational units. Unlike conventional machine learning, deep learning autonomously learns from the environment and past errors without human intervention. This enables deep learning models to establish nonlinear correlations, resulting in higher accuracy. Recent advancements in deep learning have led to the development of transformers, which are particularly effective in learning text representations. Noteworthy examples include bidirectional encoder representations from transformers (BERT) (Devlin et al. 2019 ) and the generative pretrained transformer (GPT) (OpenAI).

BERT is a linguistic representation model that utilizes a transformer architecture and is trained on two tasks: masked linguistic modeling and next-sentence prediction (Hirao et al. 2020 ; Vaswani et al. 2017 ). In the context of AES, BERT follows specific procedures, as illustrated in Fig. 1 : (a) the tokenized prompts and essays are taken as input; (b) special tokens, such as [CLS] and [SEP], are added to mark the beginning and separation of prompts and essays; (c) the transformer encoder processes the prompt and essay sequences, resulting in hidden layer sequences; (d) the hidden layers corresponding to the [CLS] tokens (T[CLS]) represent distributed representations of the prompts and essays; and (e) a multilayer perceptron uses these distributed representations as input to obtain the final score (Hirao et al. 2020 ).

AES system with BERT (Hirao et al. 2020 ).

The training of BERT using a substantial amount of sentence data through the Masked Language Model (MLM) allows it to capture contextual information within the hidden layers. Consequently, BERT is expected to be capable of identifying artificial essays as invalid and assigning them lower scores (Mizumoto and Eguchi, 2023 ). In the context of AES for nonnative Japanese learners, Hirao et al. ( 2020 ) combined the long short-term memory (LSTM) model proposed by Hochreiter and Schmidhuber ( 1997 ) with BERT to develop a tailored automated Essay Scoring System. The findings of their study revealed that the BERT model outperformed both the conventional machine learning approach utilizing character-type features such as “kanji” and “hiragana”, as well as the standalone LSTM model. Takeuchi et al. ( 2021 ) presented an approach to Japanese AES that eliminates the requirement for pre-scored essays by relying solely on reference texts or a model answer for the essay task. They investigated multiple similarity evaluation methods, including frequency of morphemes, idf values calculated on Wikipedia, LSI, LDA, word-embedding vectors, and document vectors produced by BERT. The experimental findings revealed that the method utilizing the frequency of morphemes with idf values exhibited the strongest correlation with human-annotated scores across different essay tasks. The utilization of BERT in AES encounters several limitations. Firstly, essays often exceed the model’s maximum length limit. Second, only score labels are available for training, which restricts access to additional information.

Mizumoto and Eguchi ( 2023 ) were pioneers in employing the GPT model for AES in non-native English writing. Their study focused on evaluating the accuracy and reliability of AES using the GPT-3 text-davinci-003 model, analyzing a dataset of 12,100 essays from the corpus of nonnative written English (TOEFL11). The findings indicated that AES utilizing the GPT-3 model exhibited a certain degree of accuracy and reliability. They suggest that GPT-3-based AES systems hold the potential to provide support for human ratings. However, applying GPT model to AES presents a unique natural language processing (NLP) task that involves considerations such as nonnative language proficiency, the influence of the learner’s first language on the output in the target language, and identifying linguistic features that best indicate writing quality in a specific language. These linguistic features may differ morphologically or syntactically from those present in the learners’ first language, as observed in (1)–(3).

我-送了-他-一本-书

Wǒ-sòngle-tā-yī běn-shū

1 sg .-give. past- him-one .cl- book

“I gave him a book.”

Agglutinative

彼-に-本-を-あげ-まし-た

Kare-ni-hon-o-age-mashi-ta

3 sg .- dat -hon- acc- give.honorification. past

Inflectional

give, give-s, gave, given, giving

Additionally, the morphological agglutination and subject-object-verb (SOV) order in Japanese, along with its idiomatic expressions, pose additional challenges for applying language models in AES tasks (4).

足-が棒-になり-ました

Ashi-ga bo-ni nar-mashita

leg- nom stick- dat become- past

“My leg became like a stick (I am extremely tired).”

The example sentence provided demonstrates the morpho-syntactic structure of Japanese and the presence of an idiomatic expression. In this sentence, the verb “なる” (naru), meaning “to become”, appears at the end of the sentence. The verb stem “なり” (nari) is attached with morphemes indicating honorification (“ます” - mashu) and tense (“た” - ta), showcasing agglutination. While the sentence can be literally translated as “my leg became like a stick”, it carries an idiomatic interpretation that implies “I am extremely tired”.

To overcome this issue, CyberAgent Inc. ( 2023 ) has developed the Open-Calm series of language models specifically designed for Japanese. Open-Calm consists of pre-trained models available in various sizes, such as Small, Medium, Large, and 7b. Figure 2 depicts the fundamental structure of the Open-Calm model. A key feature of this architecture is the incorporation of the Lora Adapter and GPT-NeoX frameworks, which can enhance its language processing capabilities.

GPT-NeoX Model Architecture (Okgetheng and Takeuchi 2024 ).

In a recent study conducted by Okgetheng and Takeuchi ( 2024 ), they assessed the efficacy of Open-Calm language models in grading Japanese essays. The research utilized a dataset of approximately 300 essays, which were annotated by native Japanese educators. The findings of the study demonstrate the considerable potential of Open-Calm language models in automated Japanese essay scoring. Specifically, among the Open-Calm family, the Open-Calm Large model (referred to as OCLL) exhibited the highest performance. However, it is important to note that, as of the current date, the Open-Calm Large model does not offer public access to its server. Consequently, users are required to independently deploy and operate the environment for OCLL. In order to utilize OCLL, users must have a PC equipped with an NVIDIA GeForce RTX 3060 (8 or 12 GB VRAM).

In summary, while the potential of LLMs in automated scoring of nonnative Japanese essays has been demonstrated in two studies—BERT-driven AES (Hirao et al. 2020 ) and OCLL-based AES (Okgetheng and Takeuchi, 2024 )—the number of research efforts in this area remains limited.

Another significant challenge in applying LLMs to AES lies in prompt engineering and ensuring its reliability and effectiveness (Brown et al. 2020 ; Rae et al. 2021 ; Zhang et al. 2021 ). Various prompting strategies have been proposed, such as the zero-shot chain of thought (CoT) approach (Kojima et al. 2022 ), which involves manually crafting diverse and effective examples. However, manual efforts can lead to mistakes. To address this, Zhang et al. ( 2021 ) introduced an automatic CoT prompting method called Auto-CoT, which demonstrates matching or superior performance compared to the CoT paradigm. Another prompt framework is trees of thoughts, enabling a model to self-evaluate its progress at intermediate stages of problem-solving through deliberate reasoning (Yao et al. 2023 ).

Beyond linguistic studies, there has been a noticeable increase in the number of foreign workers in Japan and Japanese learners worldwide (Ministry of Health, Labor, and Welfare of Japan, 2022 ; Japan Foundation, 2021 ). However, existing assessment methods, such as the Japanese Language Proficiency Test (JLPT), J-CAT, and TTBJ Footnote 1 , primarily focus on reading, listening, vocabulary, and grammar skills, neglecting the evaluation of writing proficiency. As the number of workers and language learners continues to grow, there is a rising demand for an efficient AES system that can reduce costs and time for raters and be utilized for employment, examinations, and self-study purposes.

This study aims to explore the potential of LLM-based AES by comparing the effectiveness of five models: two LLMs (GPT Footnote 2 and BERT), one Japanese local LLM (OCLL), and two conventional machine learning-based methods (linguistic feature-based scoring tools - Jess and JWriter).

The research questions addressed in this study are as follows:

To what extent do the LLM-driven AES and linguistic feature-based AES, when used as automated tools to support human rating, accurately reflect test takers’ actual performance?

What influence does the prompt have on the accuracy and performance of LLM-based AES methods?

The subsequent sections of the manuscript cover the methodology, including the assessment measures for nonnative Japanese writing proficiency, criteria for prompts, and the dataset. The evaluation section focuses on the analysis of annotations and rating scores generated by LLM-driven and linguistic feature-based AES methods.

Methodology

The dataset utilized in this study was obtained from the International Corpus of Japanese as a Second Language (I-JAS) Footnote 3 . This corpus consisted of 1000 participants who represented 12 different first languages. For the study, the participants were given a story-writing task on a personal computer. They were required to write two stories based on the 4-panel illustrations titled “Picnic” and “The key” (see Appendix A). Background information for the participants was provided by the corpus, including their Japanese language proficiency levels assessed through two online tests: J-CAT and SPOT. These tests evaluated their reading, listening, vocabulary, and grammar abilities. The learners’ proficiency levels were categorized into six levels aligned with the Common European Framework of Reference for Languages (CEFR) and the Reference Framework for Japanese Language Education (RFJLE): A1, A2, B1, B2, C1, and C2. According to Lee et al. ( 2015 ), there is a high level of agreement (r = 0.86) between the J-CAT and SPOT assessments, indicating that the proficiency certifications provided by J-CAT are consistent with those of SPOT. However, it is important to note that the scores of J-CAT and SPOT do not have a one-to-one correspondence. In this study, the J-CAT scores were used as a benchmark to differentiate learners of different proficiency levels. A total of 1400 essays were utilized, representing the beginner (aligned with A1), A2, B1, B2, C1, and C2 levels based on the J-CAT scores. Table 1 provides information about the learners’ proficiency levels and their corresponding J-CAT and SPOT scores.

A dataset comprising a total of 1400 essays from the story writing tasks was collected. Among these, 714 essays were utilized to evaluate the reliability of the LLM-based AES method, while the remaining 686 essays were designated as development data to assess the LLM-based AES’s capability to distinguish participants with varying proficiency levels. The GPT 4 API was used in this study. A detailed explanation of the prompt-assessment criteria is provided in Section Prompt . All essays were sent to the model for measurement and scoring.

Measures of writing proficiency for nonnative Japanese

Japanese exhibits a morphologically agglutinative structure where morphemes are attached to the word stem to convey grammatical functions such as tense, aspect, voice, and honorifics, e.g. (5).

食べ-させ-られ-まし-た-か

tabe-sase-rare-mashi-ta-ka

[eat (stem)-causative-passive voice-honorification-tense. past-question marker]

Japanese employs nine case particles to indicate grammatical functions: the nominative case particle が (ga), the accusative case particle を (o), the genitive case particle の (no), the dative case particle に (ni), the locative/instrumental case particle で (de), the ablative case particle から (kara), the directional case particle へ (e), and the comitative case particle と (to). The agglutinative nature of the language, combined with the case particle system, provides an efficient means of distinguishing between active and passive voice, either through morphemes or case particles, e.g. 食べる taberu “eat concusive . ” (active voice); 食べられる taberareru “eat concusive . ” (passive voice). In the active voice, “パンを食べる” (pan o taberu) translates to “to eat bread”. On the other hand, in the passive voice, it becomes “パンが食べられた” (pan ga taberareta), which means “(the) bread was eaten”. Additionally, it is important to note that different conjugations of the same lemma are considered as one type in order to ensure a comprehensive assessment of the language features. For example, e.g., 食べる taberu “eat concusive . ”; 食べている tabeteiru “eat progress .”; 食べた tabeta “eat past . ” as one type.

To incorporate these features, previous research (Suzuki, 1999 ; Watanabe et al. 1988 ; Ishioka, 2001 ; Ishioka and Kameda, 2006 ; Hirao et al. 2020 ) has identified complexity, fluency, and accuracy as crucial factors for evaluating writing quality. These criteria are assessed through various aspects, including lexical richness (lexical density, diversity, and sophistication), syntactic complexity, and cohesion (Kyle et al. 2021 ; Mizumoto and Eguchi, 2023 ; Ure, 1971 ; Halliday, 1985 ; Barkaoui and Hadidi, 2020 ; Zenker and Kyle, 2021 ; Kim et al. 2018 ; Lu, 2017 ; Ortega, 2015 ). Therefore, this study proposes five scoring categories: lexical richness, syntactic complexity, cohesion, content elaboration, and grammatical accuracy. A total of 16 measures were employed to capture these categories. The calculation process and specific details of these measures can be found in Table 2 .

T-unit, first introduced by Hunt ( 1966 ), is a measure used for evaluating speech and composition. It serves as an indicator of syntactic development and represents the shortest units into which a piece of discourse can be divided without leaving any sentence fragments. In the context of Japanese language assessment, Sakoda and Hosoi ( 2020 ) utilized T-unit as the basic unit to assess the accuracy and complexity of Japanese learners’ speaking and storytelling. The calculation of T-units in Japanese follows the following principles:

A single main clause constitutes 1 T-unit, regardless of the presence or absence of dependent clauses, e.g. (6).

ケンとマリはピクニックに行きました (main clause): 1 T-unit.

If a sentence contains a main clause along with subclauses, each subclause is considered part of the same T-unit, e.g. (7).

天気が良かったので (subclause)、ケンとマリはピクニックに行きました (main clause): 1 T-unit.

In the case of coordinate clauses, where multiple clauses are connected, each coordinated clause is counted separately. Thus, a sentence with coordinate clauses may have 2 T-units or more, e.g. (8).

ケンは地図で場所を探して (coordinate clause)、マリはサンドイッチを作りました (coordinate clause): 2 T-units.

Lexical diversity refers to the range of words used within a text (Engber, 1995 ; Kyle et al. 2021 ) and is considered a useful measure of the breadth of vocabulary in L n production (Jarvis, 2013a , 2013b ).

The type/token ratio (TTR) is widely recognized as a straightforward measure for calculating lexical diversity and has been employed in numerous studies. These studies have demonstrated a strong correlation between TTR and other methods of measuring lexical diversity (e.g., Bentz et al. 2016 ; Čech and Miroslav, 2018 ; Çöltekin and Taraka, 2018 ). TTR is computed by considering both the number of unique words (types) and the total number of words (tokens) in a given text. Given that the length of learners’ writing texts can vary, this study employs the moving average type-token ratio (MATTR) to mitigate the influence of text length. MATTR is calculated using a 50-word moving window. Initially, a TTR is determined for words 1–50 in an essay, followed by words 2–51, 3–52, and so on until the end of the essay is reached (Díez-Ortega and Kyle, 2023 ). The final MATTR scores were obtained by averaging the TTR scores for all 50-word windows. The following formula was employed to derive MATTR:

${\rm{MATTR}}({\rm{W}})=\frac{{\sum }_{{\rm{i}}=1}^{{\rm{N}}-{\rm{W}}+1}{{\rm{F}}}_{{\rm{i}}}}{{\rm{W}}({\rm{N}}-{\rm{W}}+1)}$

Here, N refers to the number of tokens in the corpus. W is the randomly selected token size (W < N). ${F}_{i}$ is the number of types in each window. The ${\rm{MATTR}}({\rm{W}})$ is the mean of a series of type-token ratios (TTRs) based on the word form for all windows. It is expected that individuals with higher language proficiency will produce texts with greater lexical diversity, as indicated by higher MATTR scores.

Lexical density was captured by the ratio of the number of lexical words to the total number of words (Lu, 2012 ). Lexical sophistication refers to the utilization of advanced vocabulary, often evaluated through word frequency indices (Crossley et al. 2013 ; Haberman, 2008 ; Kyle and Crossley, 2015 ; Laufer and Nation, 1995 ; Lu, 2012 ; Read, 2000 ). In line of writing, lexical sophistication can be interpreted as vocabulary breadth, which entails the appropriate usage of vocabulary items across various lexicon-grammatical contexts and registers (Garner et al. 2019 ; Kim et al. 2018 ; Kyle et al. 2018 ). In Japanese specifically, words are considered lexically sophisticated if they are not included in the “Japanese Education Vocabulary List Ver 1.0”. Footnote 4 Consequently, lexical sophistication was calculated by determining the number of sophisticated word types relative to the total number of words per essay. Furthermore, it has been suggested that, in Japanese writing, sentences should ideally have a length of no more than 40 to 50 characters, as this promotes readability. Therefore, the median and maximum sentence length can be considered as useful indices for assessment (Ishioka and Kameda, 2006 ).

Syntactic complexity was assessed based on several measures, including the mean length of clauses, verb phrases per T-unit, clauses per T-unit, dependent clauses per T-unit, complex nominals per clause, adverbial clauses per clause, coordinate phrases per clause, and mean dependency distance (MDD). The MDD reflects the distance between the governor and dependent positions in a sentence. A larger dependency distance indicates a higher cognitive load and greater complexity in syntactic processing (Liu, 2008 ; Liu et al. 2017 ). The MDD has been established as an efficient metric for measuring syntactic complexity (Jiang, Quyang, and Liu, 2019 ; Li and Yan, 2021 ). To calculate the MDD, the position numbers of the governor and dependent are subtracted, assuming that words in a sentence are assigned in a linear order, such as W1 … Wi … Wn. In any dependency relationship between words Wa and Wb, Wa is the governor and Wb is the dependent. The MDD of the entire sentence was obtained by taking the absolute value of governor – dependent:

MDD = $\frac{1}{n}{\sum }_{i=1}^{n}|{\rm{D}}{{\rm{D}}}_{i}|$

In this formula, $n$ represents the number of words in the sentence, and ${DD}i$ is the dependency distance of the ${i}^{{th}}$ dependency relationship of a sentence. Building on this, the annotation of sentence ‘Mary-ga-John-ni-keshigomu-o-watashita was [Mary- top -John- dat -eraser- acc -give- past] ’. The sentence’s MDD would be 2. Table 3 provides the CSV file as a prompt for GPT 4.

Cohesion (semantic similarity) and content elaboration aim to capture the ideas presented in test taker’s essays. Cohesion was assessed using three measures: Synonym overlap/paragraph (topic), Synonym overlap/paragraph (keywords), and word2vec cosine similarity. Content elaboration and development were measured as the number of metadiscourse markers (type)/number of words. To capture content closely, this study proposed a novel-distance based representation, by encoding the cosine distance between the essay (by learner) and essay task’s (topic and keyword) i -vectors. The learner’s essay is decoded into a word sequence, and aligned to the essay task’ topic and keyword for log-likelihood measurement. The cosine distance reveals the content elaboration score in the leaners’ essay. The mathematical equation of cosine similarity between target-reference vectors is shown in (11), assuming there are i essays and ( L i , …. L n ) and ( N i , …. N n ) are the vectors representing the learner and task’s topic and keyword respectively. The content elaboration distance between L i and N i was calculated as follows:

$\cos \left(\theta \right)=\frac{{\rm{L}}\,\cdot\, {\rm{N}}}{\left|{\rm{L}}\right|{\rm{|N|}}}=\frac{\mathop{\sum }\nolimits_{i=1}^{n}{L}_{i}{N}_{i}}{\sqrt{\mathop{\sum }\nolimits_{i=1}^{n}{L}_{i}^{2}}\sqrt{\mathop{\sum }\nolimits_{i=1}^{n}{N}_{i}^{2}}}$

A high similarity value indicates a low difference between the two recognition outcomes, which in turn suggests a high level of proficiency in content elaboration.

To evaluate the effectiveness of the proposed measures in distinguishing different proficiency levels among nonnative Japanese speakers’ writing, we conducted a multi-faceted Rasch measurement analysis (Linacre, 1994 ). This approach applies measurement models to thoroughly analyze various factors that can influence test outcomes, including test takers’ proficiency, item difficulty, and rater severity, among others. The underlying principles and functionality of multi-faceted Rasch measurement are illustrated in (12).

$\log \left(\frac{{P}_{{nijk}}}{{P}_{{nij}(k-1)}}\right)={B}_{n}-{D}_{i}-{C}_{j}-{F}_{k}$

(12) defines the logarithmic transformation of the probability ratio ( P nijk /P nij(k-1) )) as a function of multiple parameters. Here, n represents the test taker, i denotes a writing proficiency measure, j corresponds to the human rater, and k represents the proficiency score. The parameter B n signifies the proficiency level of test taker n (where n ranges from 1 to N). D j represents the difficulty parameter of test item i (where i ranges from 1 to L), while C j represents the severity of rater j (where j ranges from 1 to J). Additionally, F k represents the step difficulty for a test taker to move from score ‘k-1’ to k . P nijk refers to the probability of rater j assigning score k to test taker n for test item i . P nij(k-1) represents the likelihood of test taker n being assigned score ‘k-1’ by rater j for test item i . Each facet within the test is treated as an independent parameter and estimated within the same reference framework. To evaluate the consistency of scores obtained through both human and computer analysis, we utilized the Infit mean-square statistic. This statistic is a chi-square measure divided by the degrees of freedom and is weighted with information. It demonstrates higher sensitivity to unexpected patterns in responses to items near a person’s proficiency level (Linacre, 2002 ). Fit statistics are assessed based on predefined thresholds for acceptable fit. For the Infit MNSQ, which has a mean of 1.00, different thresholds have been suggested. Some propose stricter thresholds ranging from 0.7 to 1.3 (Bond et al. 2021 ), while others suggest more lenient thresholds ranging from 0.5 to 1.5 (Eckes, 2009 ). In this study, we adopted the criterion of 0.70–1.30 for the Infit MNSQ.

Moving forward, we can now proceed to assess the effectiveness of the 16 proposed measures based on five criteria for accurately distinguishing various levels of writing proficiency among non-native Japanese speakers. To conduct this evaluation, we utilized the development dataset from the I-JAS corpus, as described in Section Dataset . Table 4 provides a measurement report that presents the performance details of the 14 metrics under consideration. The measure separation was found to be 4.02, indicating a clear differentiation among the measures. The reliability index for the measure separation was 0.891, suggesting consistency in the measurement. Similarly, the person separation reliability index was 0.802, indicating the accuracy of the assessment in distinguishing between individuals. All 16 measures demonstrated Infit mean squares within a reasonable range, ranging from 0.76 to 1.28. The Synonym overlap/paragraph (topic) measure exhibited a relatively high outfit mean square of 1.46, although the Infit mean square falls within an acceptable range. The standard error for the measures ranged from 0.13 to 0.28, indicating the precision of the estimates.

Table 5 further illustrated the weights assigned to different linguistic measures for score prediction, with higher weights indicating stronger correlations between those measures and higher scores. Specifically, the following measures exhibited higher weights compared to others: moving average type token ratio per essay has a weight of 0.0391. Mean dependency distance had a weight of 0.0388. Mean length of clause, calculated by dividing the number of words by the number of clauses, had a weight of 0.0374. Complex nominals per T-unit, calculated by dividing the number of complex nominals by the number of T-units, had a weight of 0.0379. Coordinate phrases rate, calculated by dividing the number of coordinate phrases by the number of clauses, had a weight of 0.0325. Grammatical error rate, representing the number of errors per essay, had a weight of 0.0322.

Criteria (output indicator)

The criteria used to evaluate the writing ability in this study were based on CEFR, which follows a six-point scale ranging from A1 to C2. To assess the quality of Japanese writing, the scoring criteria from Table 6 were utilized. These criteria were derived from the IELTS writing standards and served as assessment guidelines and prompts for the written output.

A prompt is a question or detailed instruction that is provided to the model to obtain a proper response. After several pilot experiments, we decided to provide the measures (Section Measures of writing proficiency for nonnative Japanese ) as the input prompt and use the criteria (Section Criteria (output indicator) ) as the output indicator. Regarding the prompt language, considering that the LLM was tasked with rating Japanese essays, would prompt in Japanese works better Footnote 5 ? We conducted experiments comparing the performance of GPT-4 using both English and Japanese prompts. Additionally, we utilized the Japanese local model OCLL with Japanese prompts. Multiple trials were conducted using the same sample. Regardless of the prompt language used, we consistently obtained the same grading results with GPT-4, which assigned a grade of B1 to the writing sample. This suggested that GPT-4 is reliable and capable of producing consistent ratings regardless of the prompt language. On the other hand, when we used Japanese prompts with the Japanese local model “OCLL”, we encountered inconsistent grading results. Out of 10 attempts with OCLL, only 6 yielded consistent grading results (B1), while the remaining 4 showed different outcomes, including A1 and B2 grades. These findings indicated that the language of the prompt was not the determining factor for reliable AES. Instead, the size of the training data and the model parameters played crucial roles in achieving consistent and reliable AES results for the language model.

The following is the utilized prompt, which details all measures and requires the LLM to score the essays using holistic and trait scores.

Please evaluate Japanese essays written by Japanese learners and assign a score to each essay on a six-point scale, ranging from A1, A2, B1, B2, C1 to C2. Additionally, please provide trait scores and display the calculation process for each trait score. The scoring should be based on the following criteria:

Moving average type-token ratio.

Number of lexical words (token) divided by the total number of words per essay.

Number of sophisticated word types divided by the total number of words per essay.

Mean length of clause.

Verb phrases per T-unit.

Clauses per T-unit.

Dependent clauses per T-unit.

Complex nominals per clause.

Adverbial clauses per clause.

Coordinate phrases per clause.

Mean dependency distance.

Synonym overlap paragraph (topic and keywords).

Word2vec cosine similarity.

Connectives per essay.

Conjunctions per essay.

Number of metadiscourse markers (types) divided by the total number of words.

Number of errors per essay.

Japanese essay text

出かける前に二人が地図を見ている間に、サンドイッチを入れたバスケットに犬が入ってしまいました。それに気づかずに二人は楽しそうに出かけて行きました。やがて突然犬がバスケットから飛び出し、二人は驚きました。バスケットの中を見ると、食べ物はすべて犬に食べられていて、二人は困ってしまいました。(ID_JJJ01_SW1)

The score of the example above was B1. Figure 3 provides an example of holistic and trait scores provided by GPT-4 (with a prompt indicating all measures) via Bing Footnote 6 .

Example of GPT-4 AES and feedback (with a prompt indicating all measures).

Statistical analysis

The aim of this study is to investigate the potential use of LLM for nonnative Japanese AES. It seeks to compare the scoring outcomes obtained from feature-based AES tools, which rely on conventional machine learning technology (i.e. Jess, JWriter), with those generated by AI-driven AES tools utilizing deep learning technology (BERT, GPT, OCLL). To assess the reliability of a computer-assisted annotation tool, the study initially established human-human agreement as the benchmark measure. Subsequently, the performance of the LLM-based method was evaluated by comparing it to human-human agreement.

To assess annotation agreement, the study employed standard measures such as precision, recall, and F-score (Brants 2000 ; Lu 2010 ), along with the quadratically weighted kappa (QWK) to evaluate the consistency and agreement in the annotation process. Assume A and B represent human annotators. When comparing the annotations of the two annotators, the following results are obtained. The evaluation of precision, recall, and F-score metrics was illustrated in equations (13) to (15).

${\rm{Recall}}(A,B)=\frac{{\rm{Number}}\,{\rm{of}}\,{\rm{identical}}\,{\rm{nodes}}\,{\rm{in}}\,A\,{\rm{and}}\,B}{{\rm{Number}}\,{\rm{of}}\,{\rm{nodes}}\,{\rm{in}}\,A}$

${\rm{Precision}}(A,\,B)=\frac{{\rm{Number}}\,{\rm{of}}\,{\rm{identical}}\,{\rm{nodes}}\,{\rm{in}}\,A\,{\rm{and}}\,B}{{\rm{Number}}\,{\rm{of}}\,{\rm{nodes}}\,{\rm{in}}\,B}$

The F-score is the harmonic mean of recall and precision:

${\rm{F}}-{\rm{score}}=\frac{2* ({\rm{Precision}}* {\rm{Recall}})}{{\rm{Precision}}+{\rm{Recall}}}$

The highest possible value of an F-score is 1.0, indicating perfect precision and recall, and the lowest possible value is 0, if either precision or recall are zero.

In accordance with Taghipour and Ng ( 2016 ), the calculation of QWK involves two steps:

Step 1: Construct a weight matrix W as follows:

${W}_{{ij}}=\frac{{(i-j)}^{2}}{{(N-1)}^{2}}$

i represents the annotation made by the tool, while j represents the annotation made by a human rater. N denotes the total number of possible annotations. Matrix O is subsequently computed, where O_( i, j ) represents the count of data annotated by the tool ( i ) and the human annotator ( j ). On the other hand, E refers to the expected count matrix, which undergoes normalization to ensure that the sum of elements in E matches the sum of elements in O.

Step 2: With matrices O and E, the QWK is obtained as follows:

K = 1- $\frac{\sum i,j{W}_{i,j}\,{O}_{i,j}}{\sum i,j{W}_{i,j}\,{E}_{i,j}}$

The value of the quadratic weighted kappa increases as the level of agreement improves. Further, to assess the accuracy of LLM scoring, the proportional reductive mean square error (PRMSE) was employed. The PRMSE approach takes into account the variability observed in human ratings to estimate the rater error, which is then subtracted from the variance of the human labels. This calculation provides an overall measure of agreement between the automated scores and true scores (Haberman et al. 2015 ; Loukina et al. 2020 ; Taghipour and Ng, 2016 ). The computation of PRMSE involves the following steps:

Step 1: Calculate the mean squared errors (MSEs) for the scoring outcomes of the computer-assisted tool (MSE tool) and the human scoring outcomes (MSE human).

Step 2: Determine the PRMSE by comparing the MSE of the computer-assisted tool (MSE tool) with the MSE from human raters (MSE human), using the following formula:

${\rm{PRMSE}}=1-\frac{({\rm{MSE}}\,{\rm{tool}})\,}{({\rm{MSE}}\,{\rm{human}})\,}=1-\,\frac{{\sum }_{i}^{n}=1{({{\rm{y}}}_{i}-{\hat{{\rm{y}}}}_{{\rm{i}}})}^{2}}{{\sum }_{i}^{n}=1{({{\rm{y}}}_{i}-\hat{{\rm{y}}})}^{2}}$

In the numerator, ŷi represents the scoring outcome predicted by a specific LLM-driven AES system for a given sample. The term y i − ŷ i represents the difference between this predicted outcome and the mean value of all LLM-driven AES systems’ scoring outcomes. It quantifies the deviation of the specific LLM-driven AES system’s prediction from the average prediction of all LLM-driven AES systems. In the denominator, y i − ŷ represents the difference between the scoring outcome provided by a specific human rater for a given sample and the mean value of all human raters’ scoring outcomes. It measures the discrepancy between the specific human rater’s score and the average score given by all human raters. The PRMSE is then calculated by subtracting the ratio of the MSE tool to the MSE human from 1. PRMSE falls within the range of 0 to 1, with larger values indicating reduced errors in LLM’s scoring compared to those of human raters. In other words, a higher PRMSE implies that LLM’s scoring demonstrates greater accuracy in predicting the true scores (Loukina et al. 2020 ). The interpretation of kappa values, ranging from 0 to 1, is based on the work of Landis and Koch ( 1977 ). Specifically, the following categories are assigned to different ranges of kappa values: −1 indicates complete inconsistency, 0 indicates random agreement, 0.0 ~ 0.20 indicates extremely low level of agreement (slight), 0.21 ~ 0.40 indicates moderate level of agreement (fair), 0.41 ~ 0.60 indicates medium level of agreement (moderate), 0.61 ~ 0.80 indicates high level of agreement (substantial), 0.81 ~ 1 indicates almost perfect level of agreement. All statistical analyses were executed using Python script.

Results and discussion

Annotation reliability of the llm.

This section focuses on assessing the reliability of the LLM’s annotation and scoring capabilities. To evaluate the reliability, several tests were conducted simultaneously, aiming to achieve the following objectives:

Assess the LLM’s ability to differentiate between test takers with varying levels of oral proficiency.

Determine the level of agreement between the annotations and scoring performed by the LLM and those done by human raters.

The evaluation of the results encompassed several metrics, including: precision, recall, F-Score, quadratically-weighted kappa, proportional reduction of mean squared error, Pearson correlation, and multi-faceted Rasch measurement.

Inter-annotator agreement (human–human annotator agreement)

We started with an agreement test of the two human annotators. Two trained annotators were recruited to determine the writing task data measures. A total of 714 scripts, as the test data, was utilized. Each analysis lasted 300–360 min. Inter-annotator agreement was evaluated using the standard measures of precision, recall, and F-score and QWK. Table 7 presents the inter-annotator agreement for the various indicators. As shown, the inter-annotator agreement was fairly high, with F-scores ranging from 1.0 for sentence and word number to 0.666 for grammatical errors.

The findings from the QWK analysis provided further confirmation of the inter-annotator agreement. The QWK values covered a range from 0.950 ( p = 0.000) for sentence and word number to 0.695 for synonym overlap number (keyword) and grammatical errors ( p = 0.001).

Agreement of annotation outcomes between human and LLM

To evaluate the consistency between human annotators and LLM annotators (BERT, GPT, OCLL) across the indices, the same test was conducted. The results of the inter-annotator agreement (F-score) between LLM and human annotation are provided in Appendix B-D. The F-scores ranged from 0.706 for Grammatical error # for OCLL-human to a perfect 1.000 for GPT-human, for sentences, clauses, T-units, and words. These findings were further supported by the QWK analysis, which showed agreement levels ranging from 0.807 ( p = 0.001) for metadiscourse markers for OCLL-human to 0.962 for words ( p = 0.000) for GPT-human. The findings demonstrated that the LLM annotation achieved a significant level of accuracy in identifying measurement units and counts.

Reliability of LLM-driven AES’s scoring and discriminating proficiency levels

This section examines the reliability of the LLM-driven AES scoring through a comparison of the scoring outcomes produced by human raters and the LLM ( Reliability of LLM-driven AES scoring ). It also assesses the effectiveness of the LLM-based AES system in differentiating participants with varying proficiency levels ( Reliability of LLM-driven AES discriminating proficiency levels ).

Reliability of LLM-driven AES scoring

Table 8 summarizes the QWK coefficient analysis between the scores computed by the human raters and the GPT-4 for the individual essays from I-JAS Footnote 7 . As shown, the QWK of all measures ranged from k = 0.819 for lexical density (number of lexical words (tokens)/number of words per essay) to k = 0.644 for word2vec cosine similarity. Table 9 further presents the Pearson correlations between the 16 writing proficiency measures scored by human raters and GPT 4 for the individual essays. The correlations ranged from 0.672 for syntactic complexity to 0.734 for grammatical accuracy. The correlations between the writing proficiency scores assigned by human raters and the BERT-based AES system were found to range from 0.661 for syntactic complexity to 0.713 for grammatical accuracy. The correlations between the writing proficiency scores given by human raters and the OCLL-based AES system ranged from 0.654 for cohesion to 0.721 for grammatical accuracy. These findings indicated an alignment between the assessments made by human raters and both the BERT-based and OCLL-based AES systems in terms of various aspects of writing proficiency.

Reliability of LLM-driven AES discriminating proficiency levels

After validating the reliability of the LLM’s annotation and scoring, the subsequent objective was to evaluate its ability to distinguish between various proficiency levels. For this analysis, a dataset of 686 individual essays was utilized. Table 10 presents a sample of the results, summarizing the means, standard deviations, and the outcomes of the one-way ANOVAs based on the measures assessed by the GPT-4 model. A post hoc multiple comparison test, specifically the Bonferroni test, was conducted to identify any potential differences between pairs of levels.

As the results reveal, seven measures presented linear upward or downward progress across the three proficiency levels. These were marked in bold in Table 10 and comprise one measure of lexical richness, i.e. MATTR (lexical diversity); four measures of syntactic complexity, i.e. MDD (mean dependency distance), MLC (mean length of clause), CNT (complex nominals per T-unit), CPC (coordinate phrases rate); one cohesion measure, i.e. word2vec cosine similarity and GER (grammatical error rate). Regarding the ability of the sixteen measures to distinguish adjacent proficiency levels, the Bonferroni tests indicated that statistically significant differences exist between the primary level and the intermediate level for MLC and GER. One measure of lexical richness, namely LD, along with three measures of syntactic complexity (VPT, CT, DCT, ACC), two measures of cohesion (SOPT, SOPK), and one measure of content elaboration (IMM), exhibited statistically significant differences between proficiency levels. However, these differences did not demonstrate a linear progression between adjacent proficiency levels. No significant difference was observed in lexical sophistication between proficiency levels.

To summarize, our study aimed to evaluate the reliability and differentiation capabilities of the LLM-driven AES method. For the first objective, we assessed the LLM’s ability to differentiate between test takers with varying levels of oral proficiency using precision, recall, F-Score, and quadratically-weighted kappa. Regarding the second objective, we compared the scoring outcomes generated by human raters and the LLM to determine the level of agreement. We employed quadratically-weighted kappa and Pearson correlations to compare the 16 writing proficiency measures for the individual essays. The results confirmed the feasibility of using the LLM for annotation and scoring in AES for nonnative Japanese. As a result, Research Question 1 has been addressed.

Comparison of BERT-, GPT-, OCLL-based AES, and linguistic-feature-based computation methods

This section aims to compare the effectiveness of five AES methods for nonnative Japanese writing, i.e. LLM-driven approaches utilizing BERT, GPT, and OCLL, linguistic feature-based approaches using Jess and JWriter. The comparison was conducted by comparing the ratings obtained from each approach with human ratings. All ratings were derived from the dataset introduced in Dataset . To facilitate the comparison, the agreement between the automated methods and human ratings was assessed using QWK and PRMSE. The performance of each approach was summarized in Table 11 .

The QWK coefficient values indicate that LLMs (GPT, BERT, OCLL) and human rating outcomes demonstrated higher agreement compared to feature-based AES methods (Jess and JWriter) in assessing writing proficiency criteria, including lexical richness, syntactic complexity, content, and grammatical accuracy. Among the LLMs, the GPT-4 driven AES and human rating outcomes showed the highest agreement in all criteria, except for syntactic complexity. The PRMSE values suggest that the GPT-based method outperformed linguistic feature-based methods and other LLM-based approaches. Moreover, an interesting finding emerged during the study: the agreement coefficient between GPT-4 and human scoring was even higher than the agreement between different human raters themselves. This discovery highlights the advantage of GPT-based AES over human rating. Ratings involve a series of processes, including reading the learners’ writing, evaluating the content and language, and assigning scores. Within this chain of processes, various biases can be introduced, stemming from factors such as rater biases, test design, and rating scales. These biases can impact the consistency and objectivity of human ratings. GPT-based AES may benefit from its ability to apply consistent and objective evaluation criteria. By prompting the GPT model with detailed writing scoring rubrics and linguistic features, potential biases in human ratings can be mitigated. The model follows a predefined set of guidelines and does not possess the same subjective biases that human raters may exhibit. This standardization in the evaluation process contributes to the higher agreement observed between GPT-4 and human scoring. Section Prompt strategy of the study delves further into the role of prompts in the application of LLMs to AES. It explores how the choice and implementation of prompts can impact the performance and reliability of LLM-based AES methods. Furthermore, it is important to acknowledge the strengths of the local model, i.e. the Japanese local model OCLL, which excels in processing certain idiomatic expressions. Nevertheless, our analysis indicated that GPT-4 surpasses local models in AES. This superior performance can be attributed to the larger parameter size of GPT-4, estimated to be between 500 billion and 1 trillion, which exceeds the sizes of both BERT and the local model OCLL.

Prompt strategy

In the context of prompt strategy, Mizumoto and Eguchi ( 2023 ) conducted a study where they applied the GPT-3 model to automatically score English essays in the TOEFL test. They found that the accuracy of the GPT model alone was moderate to fair. However, when they incorporated linguistic measures such as cohesion, syntactic complexity, and lexical features alongside the GPT model, the accuracy significantly improved. This highlights the importance of prompt engineering and providing the model with specific instructions to enhance its performance. In this study, a similar approach was taken to optimize the performance of LLMs. GPT-4, which outperformed BERT and OCLL, was selected as the candidate model. Model 1 was used as the baseline, representing GPT-4 without any additional prompting. Model 2, on the other hand, involved GPT-4 prompted with 16 measures that included scoring criteria, efficient linguistic features for writing assessment, and detailed measurement units and calculation formulas. The remaining models (Models 3 to 18) utilized GPT-4 prompted with individual measures. The performance of these 18 different models was assessed using the output indicators described in Section Criteria (output indicator) . By comparing the performances of these models, the study aimed to understand the impact of prompt engineering on the accuracy and effectiveness of GPT-4 in AES tasks.


Model 1: GPT-4

Model 2: GPT-4 + 17 measures

Model 3: GPT-4 + MATTR	Model 4: GPT-4 + LD	Model 5: GPT-4 + LS
Model 6: GPT-4 + MLC	Model 7: GPT-4 + VPT	Model 8: GPT-4 + CT
Model 9: GPT-4 + DCT	Model 10: GPT-4 + CNT	Model 11: GPT-4 + ACC
Model 12: GPT-4 + CPC	Model 13: GPT-4 + MDD	Model 14: GPT-4 + SOPT
Model 15: GPT-4 + SOPK	Model 16: GPT-4 + word2vec
Model 17: GPT-4 + IMM	Model 18: GPT-4 + GER

Based on the PRMSE scores presented in Fig. 4 , it was observed that Model 1, representing GPT-4 without any additional prompting, achieved a fair level of performance. However, Model 2, which utilized GPT-4 prompted with all measures, outperformed all other models in terms of PRMSE score, achieving a score of 0.681. These results indicate that the inclusion of specific measures and prompts significantly enhanced the performance of GPT-4 in AES. Among the measures, syntactic complexity was found to play a particularly significant role in improving the accuracy of GPT-4 in assessing writing quality. Following that, lexical diversity emerged as another important factor contributing to the model’s effectiveness. The study suggests that a well-prompted GPT-4 can serve as a valuable tool to support human assessors in evaluating writing quality. By utilizing GPT-4 as an automated scoring tool, the evaluation biases associated with human raters can be minimized. This has the potential to empower teachers by allowing them to focus on designing writing tasks and guiding writing strategies, while leveraging the capabilities of GPT-4 for efficient and reliable scoring.

PRMSE scores of the 18 AES models.

This study aimed to investigate two main research questions: the feasibility of utilizing LLMs for AES and the impact of prompt engineering on the application of LLMs in AES.

To address the first objective, the study compared the effectiveness of five different models: GPT, BERT, the Japanese local LLM (OCLL), and two conventional machine learning-based AES tools (Jess and JWriter). The PRMSE values indicated that the GPT-4-based method outperformed other LLMs (BERT, OCLL) and linguistic feature-based computational methods (Jess and JWriter) across various writing proficiency criteria. Furthermore, the agreement coefficient between GPT-4 and human scoring surpassed the agreement among human raters themselves, highlighting the potential of using the GPT-4 tool to enhance AES by reducing biases and subjectivity, saving time, labor, and cost, and providing valuable feedback for self-study. Regarding the second goal, the role of prompt design was investigated by comparing 18 models, including a baseline model, a model prompted with all measures, and 16 models prompted with one measure at a time. GPT-4, which outperformed BERT and OCLL, was selected as the candidate model. The PRMSE scores of the models showed that GPT-4 prompted with all measures achieved the best performance, surpassing the baseline and other models.

In conclusion, this study has demonstrated the potential of LLMs in supporting human rating in assessments. By incorporating automation, we can save time and resources while reducing biases and subjectivity inherent in human rating processes. Automated language assessments offer the advantage of accessibility, providing equal opportunities and economic feasibility for individuals who lack access to traditional assessment centers or necessary resources. LLM-based language assessments provide valuable feedback and support to learners, aiding in the enhancement of their language proficiency and the achievement of their goals. This personalized feedback can cater to individual learner needs, facilitating a more tailored and effective language-learning experience.

There are three important areas that merit further exploration. First, prompt engineering requires attention to ensure optimal performance of LLM-based AES across different language types. This study revealed that GPT-4, when prompted with all measures, outperformed models prompted with fewer measures. Therefore, investigating and refining prompt strategies can enhance the effectiveness of LLMs in automated language assessments. Second, it is crucial to explore the application of LLMs in second-language assessment and learning for oral proficiency, as well as their potential in under-resourced languages. Recent advancements in self-supervised machine learning techniques have significantly improved automatic speech recognition (ASR) systems, opening up new possibilities for creating reliable ASR systems, particularly for under-resourced languages with limited data. However, challenges persist in the field of ASR. First, ASR assumes correct word pronunciation for automatic pronunciation evaluation, which proves challenging for learners in the early stages of language acquisition due to diverse accents influenced by their native languages. Accurately segmenting short words becomes problematic in such cases. Second, developing precise audio-text transcriptions for languages with non-native accented speech poses a formidable task. Last, assessing oral proficiency levels involves capturing various linguistic features, including fluency, pronunciation, accuracy, and complexity, which are not easily captured by current NLP technology.

Data availability

The dataset utilized was obtained from the International Corpus of Japanese as a Second Language (I-JAS). The data URLs: [ https://www2.ninjal.ac.jp/jll/lsaj/ihome2.html ].

J-CAT and TTBJ are two computerized adaptive tests used to assess Japanese language proficiency.

SPOT is a specific component of the TTBJ test.

J-CAT: https://www.j-cat2.org/html/ja/pages/interpret.html

SPOT: https://ttbj.cegloc.tsukuba.ac.jp/p1.html#SPOT .

The study utilized a prompt-based GPT-4 model, developed by OpenAI, which has an impressive architecture with 1.8 trillion parameters across 120 layers. GPT-4 was trained on a vast dataset of 13 trillion tokens, using two stages: initial training on internet text datasets to predict the next token, and subsequent fine-tuning through reinforcement learning from human feedback.

https://www2.ninjal.ac.jp/jll/lsaj/ihome2-en.html .

http://jhlee.sakura.ne.jp/JEV/ by Japanese Learning Dictionary Support Group 2015.

We express our sincere gratitude to the reviewer for bringing this matter to our attention.

On February 7, 2023, Microsoft began rolling out a major overhaul to Bing that included a new chatbot feature based on OpenAI’s GPT-4 (Bing.com).

Appendix E-F present the analysis results of the QWK coefficient between the scores computed by the human raters and the BERT, OCLL models.

Attali Y, Burstein J (2006) Automated essay scoring with e-rater® V.2. J. Technol., Learn. Assess., 4

Barkaoui K, Hadidi A (2020) Assessing Change in English Second Language Writing Performance (1st ed.). Routledge, New York. https://doi.org/10.4324/9781003092346

Bentz C, Tatyana R, Koplenig A, Tanja S (2016) A comparison between morphological complexity. measures: Typological data vs. language corpora. In Proceedings of the workshop on computational linguistics for linguistic complexity (CL4LC), 142–153. Osaka, Japan: The COLING 2016 Organizing Committee

Bond TG, Yan Z, Heene M (2021) Applying the Rasch model: Fundamental measurement in the human sciences (4th ed). Routledge

Brants T (2000) Inter-annotator agreement for a German newspaper corpus. Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00), Athens, Greece, 31 May-2 June, European Language Resources Association

Brown TB, Mann B, Ryder N, et al. (2020) Language models are few-shot learners. Advances in Neural Information Processing Systems, Online, 6–12 December, Curran Associates, Inc., Red Hook, NY

Burstein J (2003) The E-rater scoring engine: Automated essay scoring with natural language processing. In Shermis MD and Burstein JC (ed) Automated Essay Scoring: A Cross-Disciplinary Perspective. Lawrence Erlbaum Associates, Mahwah, NJ

Čech R, Miroslav K (2018) Morphological richness of text. In Masako F, Václav C (ed) Taming the corpus: From inflection and lexis to interpretation, 63–77. Cham, Switzerland: Springer Nature

Çöltekin Ç, Taraka, R (2018) Exploiting Universal Dependencies treebanks for measuring morphosyntactic complexity. In Aleksandrs B, Christian B (ed), Proceedings of first workshop on measuring language complexity, 1–7. Torun, Poland

Crossley SA, Cobb T, McNamara DS (2013) Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System 41:965–981. https://doi.org/10.1016/j.system.2013.08.002

Article Google Scholar

Crossley SA, McNamara DS (2016) Say more and be more coherent: How text elaboration and cohesion can increase writing quality. J. Writ. Res. 7:351–370

CyberAgent Inc (2023) Open-Calm series of Japanese language models. Retrieved from: https://www.cyberagent.co.jp/news/detail/id=28817

Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, Minnesota, 2–7 June, pp. 4171–4186. Association for Computational Linguistics

Diez-Ortega M, Kyle K (2023) Measuring the development of lexical richness of L2 Spanish: a longitudinal learner corpus study. Studies in Second Language Acquisition 1-31

Eckes T (2009) On common ground? How raters perceive scoring criteria in oral proficiency testing. In Brown A, Hill K (ed) Language testing and evaluation 13: Tasks and criteria in performance assessment (pp. 43–73). Peter Lang Publishing

Elliot S (2003) IntelliMetric: from here to validity. In: Shermis MD, Burstein JC (ed) Automated Essay Scoring: A Cross-Disciplinary Perspective. Lawrence Erlbaum Associates, Mahwah, NJ

Google Scholar

Engber CA (1995) The relationship of lexical proficiency to the quality of ESL compositions. J. Second Lang. Writ. 4:139–155

Garner J, Crossley SA, Kyle K (2019) N-gram measures and L2 writing proficiency. System 80:176–187. https://doi.org/10.1016/j.system.2018.12.001

Haberman SJ (2008) When can subscores have value? J. Educat. Behav. Stat., 33:204–229

Haberman SJ, Yao L, Sinharay S (2015) Prediction of true test scores from observed item scores and ancillary data. Brit. J. Math. Stat. Psychol. 68:363–385

Halliday MAK (1985) Spoken and Written Language. Deakin University Press, Melbourne, Australia

Hirao R, Arai M, Shimanaka H et al. (2020) Automated essay scoring system for nonnative Japanese learners. Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pp. 1250–1257. European Language Resources Association

Hunt KW (1966) Recent Measures in Syntactic Development. Elementary English, 43(7), 732–739. http://www.jstor.org/stable/41386067

Ishioka T (2001) About e-rater, a computer-based automatic scoring system for essays [Konpyūta ni yoru essei no jidō saiten shisutemu e − rater ni tsuite]. University Entrance Examination. Forum [Daigaku nyūshi fōramu] 24:71–76

Hochreiter S, Schmidhuber J (1997) Long short- term memory. Neural Comput. 9(8):1735–1780

Article CAS PubMed Google Scholar

Ishioka T, Kameda M (2006) Automated Japanese essay scoring system based on articles written by experts. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, 17–18 July 2006, pp. 233-240. Association for Computational Linguistics, USA

Japan Foundation (2021) Retrieved from: https://www.jpf.gp.jp/j/project/japanese/survey/result/dl/survey2021/all.pdf

Jarvis S (2013a) Defining and measuring lexical diversity. In Jarvis S, Daller M (ed) Vocabulary knowledge: Human ratings and automated measures (Vol. 47, pp. 13–44). John Benjamins. https://doi.org/10.1075/sibil.47.03ch1

Jarvis S (2013b) Capturing the diversity in lexical diversity. Lang. Learn. 63:87–106. https://doi.org/10.1111/j.1467-9922.2012.00739.x

Jiang J, Quyang J, Liu H (2019) Interlanguage: A perspective of quantitative linguistic typology. Lang. Sci. 74:85–97

Kim M, Crossley SA, Kyle K (2018) Lexical sophistication as a multidimensional phenomenon: Relations to second language lexical proficiency, development, and writing quality. Mod. Lang. J. 102(1):120–141. https://doi.org/10.1111/modl.12447

Kojima T, Gu S, Reid M et al. (2022) Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, New Orleans, LA, 29 November-1 December, Curran Associates, Inc., Red Hook, NY

Kyle K, Crossley SA (2015) Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Q 49:757–786

Kyle K, Crossley SA, Berger CM (2018) The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behav. Res. Methods 50:1030–1046. https://doi.org/10.3758/s13428-017-0924-4

Article PubMed Google Scholar

Kyle K, Crossley SA, Jarvis S (2021) Assessing the validity of lexical diversity using direct judgements. Lang. Assess. Q. 18:154–170. https://doi.org/10.1080/15434303.2020.1844205

Landauer TK, Laham D, Foltz PW (2003) Automated essay scoring and annotation of essays with the Intelligent Essay Assessor. In Shermis MD, Burstein JC (ed), Automated Essay Scoring: A Cross-Disciplinary Perspective. Lawrence Erlbaum Associates, Mahwah, NJ

Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 159–174

Laufer B, Nation P (1995) Vocabulary size and use: Lexical richness in L2 written production. Appl. Linguist. 16:307–322. https://doi.org/10.1093/applin/16.3.307

Lee J, Hasebe Y (2017) jWriter Learner Text Evaluator, URL: https://jreadability.net/jwriter/

Lee J, Kobayashi N, Sakai T, Sakota K (2015) A Comparison of SPOT and J-CAT Based on Test Analysis [Tesuto bunseki ni motozuku ‘SPOT’ to ‘J-CAT’ no hikaku]. Research on the Acquisition of Second Language Japanese [Dainigengo to shite no nihongo no shūtoku kenkyū] (18) 53–69

Li W, Yan J (2021) Probability distribution of dependency distance based on a Treebank of. Japanese EFL Learners’ Interlanguage. J. Quant. Linguist. 28(2):172–186. https://doi.org/10.1080/09296174.2020.1754611

Article MathSciNet Google Scholar

Linacre JM (2002) Optimizing rating scale category effectiveness. J. Appl. Meas. 3(1):85–106

PubMed Google Scholar

Linacre JM (1994) Constructing measurement with a Many-Facet Rasch Model. In Wilson M (ed) Objective measurement: Theory into practice, Volume 2 (pp. 129–144). Norwood, NJ: Ablex

Liu H (2008) Dependency distance as a metric of language comprehension difficulty. J. Cognitive Sci. 9:159–191

Liu H, Xu C, Liang J (2017) Dependency distance: A new perspective on syntactic patterns in natural languages. Phys. Life Rev. 21. https://doi.org/10.1016/j.plrev.2017.03.002

Loukina A, Madnani N, Cahill A, et al. (2020) Using PRMSE to evaluate automated scoring systems in the presence of label noise. Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, Seattle, WA, USA → Online, 10 July, pp. 18–29. Association for Computational Linguistics

Lu X (2010) Automatic analysis of syntactic complexity in second language writing. Int. J. Corpus Linguist. 15:474–496

Lu X (2012) The relationship of lexical richness to the quality of ESL learners’ oral narratives. Mod. Lang. J. 96:190–208

Lu X (2017) Automated measurement of syntactic complexity in corpus-based L2 writing research and implications for writing assessment. Lang. Test. 34:493–511

Lu X, Hu R (2022) Sense-aware lexical sophistication indices and their relationship to second language writing quality. Behav. Res. Method. 54:1444–1460. https://doi.org/10.3758/s13428-021-01675-6

Ministry of Health, Labor, and Welfare of Japan (2022) Retrieved from: https://www.mhlw.go.jp/stf/newpage_30367.html

Mizumoto A, Eguchi M (2023) Exploring the potential of using an AI language model for automated essay scoring. Res. Methods Appl. Linguist. 3:100050

Okgetheng B, Takeuchi K (2024) Estimating Japanese Essay Grading Scores with Large Language Models. Proceedings of 30th Annual Conference of the Language Processing Society in Japan, March 2024

Ortega L (2015) Second language learning explained? SLA across 10 contemporary theories. In VanPatten B, Williams J (ed) Theories in Second Language Acquisition: An Introduction

Rae JW, Borgeaud S, Cai T, et al. (2021) Scaling Language Models: Methods, Analysis & Insights from Training Gopher. ArXiv, abs/2112.11446

Read J (2000) Assessing vocabulary. Cambridge University Press. https://doi.org/10.1017/CBO9780511732942

Rudner LM, Liang T (2002) Automated Essay Scoring Using Bayes’ Theorem. J. Technol., Learning and Assessment, 1 (2)

Sakoda K, Hosoi Y (2020) Accuracy and complexity of Japanese Language usage by SLA learners in different learning environments based on the analysis of I-JAS, a learners’ corpus of Japanese as L2. Math. Linguist. 32(7):403–418. https://doi.org/10.24701/mathling.32.7_403

Suzuki N (1999) Summary of survey results regarding comprehensive essay questions. Final report of “Joint Research on Comprehensive Examinations for the Aim of Evaluating Applicability to Each Specialized Field of Universities” for 1996-2000 [shōronbun sōgō mondai ni kansuru chōsa kekka no gaiyō. Heisei 8 - Heisei 12-nendo daigaku no kaku senmon bun’ya e no tekisei no hyōka o mokuteki to suru sōgō shiken no arikata ni kansuru kyōdō kenkyū’ saishū hōkoku-sho]. University Entrance Examination Section Center Research and Development Department [Daigaku nyūshi sentā kenkyū kaihatsubu], 21–32

Taghipour K, Ng HT (2016) A neural approach to automated essay scoring. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 1–5 November, pp. 1882–1891. Association for Computational Linguistics

Takeuchi K, Ohno M, Motojin K, Taguchi M, Inada Y, Iizuka M, Abo T, Ueda H (2021) Development of essay scoring methods based on reference texts with construction of research-available Japanese essay data. In IPSJ J 62(9):1586–1604

Ure J (1971) Lexical density: A computational technique and some findings. In Coultard M (ed) Talking about Text. English Language Research, University of Birmingham, Birmingham, England

Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Long Beach, CA, 4–7 December, pp. 5998–6008, Curran Associates, Inc., Red Hook, NY

Watanabe H, Taira Y, Inoue Y (1988) Analysis of essay evaluation data [Shōronbun hyōka dēta no kaiseki]. Bulletin of the Faculty of Education, University of Tokyo [Tōkyōdaigaku kyōiku gakubu kiyō], Vol. 28, 143–164

Yao S, Yu D, Zhao J, et al. (2023) Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

Zenker F, Kyle K (2021) Investigating minimum text lengths for lexical diversity indices. Assess. Writ. 47:100505. https://doi.org/10.1016/j.asw.2020.100505

Zhang Y, Warstadt A, Li X, et al. (2021) When do you need billions of words of pretraining data? Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online, pp. 1112-1125. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.acl-long.90

Download references

This research was funded by National Foundation of Social Sciences (22BYY186) to Wenchao Li.

Author information

Authors and affiliations.

Department of Japanese Studies, Zhejiang University, Hangzhou, China

Department of Linguistics and Applied Linguistics, Zhejiang University, Hangzhou, China

You can also search for this author in PubMed Google Scholar

Contributions

Wenchao Li is in charge of conceptualization, validation, formal analysis, investigation, data curation, visualization and writing the draft. Haitao Liu is in charge of supervision.

Corresponding author

Correspondence to Wenchao Li .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

Ethical approval was not required as the study did not involve human participants.

Informed consent

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental material file #1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Li, W., Liu, H. Applying large language models for automated essay scoring for non-native Japanese. Humanit Soc Sci Commun 11 , 723 (2024). https://doi.org/10.1057/s41599-024-03209-9

Download citation

Received : 02 February 2024

Accepted : 16 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1057/s41599-024-03209-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

IMAGES

Social Sciences Essay Example
Essay topics
Sociology as a Science Essay
Introduction to Social Sciences (400 Words)
African-American Social Science Baseline Essay, Dr. John Henrik Clark…
(PDF) Selecting a Research Topic: Building a Path in Social Science

VIDEO

Essay on Important Role of Social Media in Education
Thus Saith the Science: C. S. Lewis on the Dangers of Scientism
USS QUESTIONS SOCIAL SCIENCE, CHAPTER-1 (ENGLISH & MALAYALAM MEDIUM)
What is Social Science? #socialsciences
SSLC SOCIAL SCIENCE
Essay on Social Media in English

COMMENTS

What is a Social Science Essay?
Social science essays also need to demonstrate an effective use of social scientific skills. Perhaps the most obvious of these skills is the ability to deploy theory and evidence in an appropriate manner (as you saw in the previous section, this is what distinguishes social scientific essay writing). However, particularly as you move on to more ...
What is a Social Science Essay?
Social science essays also need to demonstrate an effective use of social scientific skills. Perhaps the most obvious of these skills is the ability to deploy theory and evidence in an appropriate manner (as you saw in the previous section, this is what distinguishes social scientific essay writing). However, particularly as you move on to more ...
What Should Be in a Social Science Essay? Fundamentals and Essential
This blogpost is also available as a PDF download, so it can be stored on your desktop and used as a checklist before submitting your essay.. The following is a condensed overview of the most important features of social science essay writing. Its aim is to cut through the noise, and focus on the most essential (and important) elements of essay writing.
PDF Essay Writing Guide
• Problem or issue set by the question/topic: Essay topics and questions in the social sciences often have an issue (or conflict) which you need to address, explore and adopt a position on. Exploring a topic or question allows you to be more analytical and successful in your writing.
Organizing Your Social Sciences Research Paper
In Good Essay Writing: A Social Sciences Guide. Peter Redman. 4th edition. (London: Sage, 2011), pp. 63-70; Reyes, Victoria. Demystifying the Journal Article. Inside Higher Education. Structure and Writing Style. I. Structure and Approach. The introduction is the broad beginning of the paper that answers three important questions for the reader:
Organizing Your Social Sciences Research Paper
The accepted form of academic writing in the social sciences can vary considerable depending on the methodological framework and the intended audience. However, most college-level research papers require careful attention to the following stylistic elements: I. The Big Picture
PDF A Student'S Guide to Writing Social Science Essays
Give the reasons, interpret and to make plain. Illustrate: Use a figure, diagram, example or even a description to explain or clarify, thus. giving a visual impression. Justify: Support a conclusion or hypothesis by making reference to available. evidence/resources and providing a reasoned argument.
PDF School of Social Science Essay Guide
School of Social Science Essay Guide 9 social sciences, the best we can normally do is to present a persuasive case, based on evidence and reason for a particular explanation. Writing Begin drafting your essay early so that you have plenty of time to review and revise, and do further research if required. As you write, pay attention to:
A Short Handbook for writing essays in the Humanities and Social Sciences
In the text "A Short Handbook for Writing Essays in the Humanities and Social Sciences" by Salvatore and Dan Allosso the authors present a simple, easy to follow guide for students to use when organizing, planning, researching, and writing an essay. In addition to essay structure, the authors also provide help with the "basics of effective ...
Good Essay Writing: A Social Sciences Guide
Writing good essays is one of the most challenging aspects of studying in the social sciences. This simple guide provides you with proven approaches and techniques to help turn you into a well-oiled, essay-writing machine. Good Essay Writing demonstrates how to think critically and formulate your argument as well as offering water-tight ...
Writing in the Social Sciences
When writing in the social sciences, however, students must also be familiar with the goals of the discipline as these inform the discipline's writing expectations. According to Ragin (1994), the primary goal of social science research is "identifying order in the complexity of social life" (para. 1). Serving the primary goal are the ...
Peter Redman (2006) Good Essay Writing
This concise guide to writing undergraduate social science essays was first published in 1998. The book has been redesigned for the 3rd edi tion, the text has been updated and new material included on how ... social science theory, explains why and gives examples of how both can be done. Having identified and explained common errors in essays
What are the social sciences?
The social sciences are made up of lots of different disciplines and sub-disciplines, which focus on different aspects of society. The social sciences are a 'broad church', including lots of different disciplinary and sub-disciplinary areas. These include, for example, sociology, anthropology, criminology, archaeology, social policy, human ...
Writing in the Social Sciences
These OWL resources will help you write in some of the social sciences, such as social work and psychology. In this section. Social Work Literature Review Guidelines This handout provides an overview of how to write literature reviews in the field of social work. It provides a list of suggestions and examples. Subsections
PDF What is a Social Science Essay?
• Four golden rules for writing a social science essay • Why an essay is not a report, newspaper article or an exam answer In this chapter we consider what is distinctive about essay writing and, in particular, essay writing in the social sciences. To start with, we look at the structure of social science essays. 2.1 The structure of a ...
Social science
social science, any branch of academic study or science that deals with human behaviour in its social and cultural aspects. Usually included within the social sciences are cultural (or social) anthropology, sociology, psychology, political science, and economics.The discipline of historiography is regarded by many as a social science, and certain areas of historical study are almost ...
Importance of Social Science in Our Daily Life
Social science enhances our critical thinking skills, broadens our perspectives, and enriches our interactions with the world. Whether through economics, psychology, sociology, or other disciplines, social science provides the foundation for a better understanding of ourselves and the societies we inhabit. Keep in mind: This is only a sample.
2021 Top 25 Social Sciences and Human Behaviour Articles
Featuring authors from around the world, these papers highlight valuable research from an international community. Browse all Top 25 subject area collections here .
Organizing Your Social Sciences Research Assignments
Reflective writing in the applied social sciences enhances decision-making skills and academic performance in ways that can inform professional practice. The act of reflective writing creates self-awareness and understanding of others. ... However, the paper should not be a disjointed, stream-of-consciousness narrative. Reflective papers are ...
What I Have Learned from Social Science
January 1, 2021 57001. I've spent my adult life in and around social science. Academically through studying psychology and linguistics (alongside philosophy), professionally through working at SAGE for over 30 years and personally through an abiding amateur interest in various fields sometimes expressed in my own writing of books or articles.
The Public Value of the Social Sciences
The social sciences are under threat from two main sources. One is external, reflected in a global university crisis that imposes the marketization of higher education on the ancient practice of scholarship. The other, internal threat is social science's withdrawal from publicly-engaged teaching and research into the protective bunker of ...
Social Sciences
Social Sciences. Social Sciences is an international, open access journal with rapid peer-review, which publishes works from a wide range of fields, including anthropology, criminology, economics, education, geography, history, law, linguistics, political science, psychology, social policy, social work, sociology and so on.
The Public Value of the Social Sciences: An Interpretative Essay
This book argues that the social sciences, which grew out of moral philosophy in Europe in the 18th century, is facing today an unprecedented challenge to prove its relevance. ... " The Public Value of the Social Sciences: An Interpretative Essay." The Social Science Journal, 51(4), pp. 687-688. Log in via your institution.
Humanities vs. social science: exploring the dichotomy
The difference between humanities and social science isn't always immediately apparent. Although both disciplines focus on the human experience, they each view it from a unique lens. This article provides an overview of the different approaches, areas of study, and career paths for your humanities vs. social science consideration so you can pinpoint which option best aligns with your goals.
Reference examples
More than 100 reference examples and their corresponding in-text citations are presented in the seventh edition Publication Manual.Examples of the most common works that writers cite are provided on this page; additional examples are available in the Publication Manual.. To find the reference example you need, first select a category (e.g., periodicals) and then choose the appropriate type of ...
Applying large language models for automated essay scoring for non
Recent advancements in artificial intelligence (AI) have led to an increased use of large language models (LLMs) for language assessment tasks such as automated essay scoring (AES), automated ...
In the pandemic, we were told to keep 6 feet apart. There's no science
The CDC would keep its six-foot social distance recommendation in place until August 2022, with some modifications as Americans got vaccinated against the virus and officials pushed to reopen schools.