Insignificant Wranglings

Today’s Plan:

• Quick Reflection on Last Class
• Discuss Graham
• Prompt Engineering

Quick Reflection on Last Class

At some point in last class, Amber asked if I wished students were more willing to debate me in class. More combative. I cannot recall the exact conversation, only the question. My answer was “yes.”

But I wanted to circle back to that. I think that gut response is true: I do wish education involved more dialectic engagement, more back and forth, more challenge. But I think I want that because it is comfortable for me. It is something that I am pretty good at. It is familiar. It comes easy. So that is my natural, gut, heat of the moment answer.

But I want to revisit that because, while true, I don’t think it is the right answer. Not ethically. There’s a few things for me to unpack here. Most important, though, is that I stress that your generation’s general disinterest in educational debate and conflict isn’t a defect. It is a difference, but not a defect. I do not mean to mourn something cherished that we have lost. I am coping with difference, with having to change, with having to figure out how to welcome you in the process (ethics as hospitality).

Now, do I believe that certain democratic processes benefit from a populace trained in debate? Particularly folks who know how it feels to lose an argument? To develop a particular distance from ideas? Yes, yes, and yes. I’m tripping around a lot of stuff that sounds like old-school-semi-toxic-masculinity. But I am not calling for a complete distance or an instance on argument-as-objective-play. I think there’s a value in experiencing argument-as-a-happening. To feeling it. Encountering difference, especially different ideas about how the world (and the classroom) might work is an experience of cognitive dissonance. It is to feel estranged from the familiar. We need that to function in a diverse society. But arguing with someone isn’t the only way to get that experience(?).

Discuss Graham

Thoughts?

Prompt Engineering

First, I wanted to think about how Mollick structures his prompts:

You are an expert at marketing. When asked to generate slogan ideas you come up with ideas that are different from each other, clever, and interesting. You use clever word play. You try not to repeat themes or ideas. Come up with 20 ideas for marketing slogans for a new mail-order catalogue (106)

Here’s how I opened my ChatGPT session:

I am a writing professor and would like to test your abilities to help me write a 2500 word paper. The topic of the paper will be on chatGPT and racial bias and discrimination. Before we start writing the paper, I would like to know if you are familiar with several different sources I would like to include in the paper.

If you are not familiar with the source, then please let me know and I will be happy to select another one.

After we ran through 3 sources, I wrote:

My paper will have three major sections:

Section One: A Troubling History of Race and Technology
Section Two: Early Problems with Race and GPT
Section Three: What kinds of alignment and regulations might we need to make AI technologies safer for black people

Along the way I’ve used a lot of prompts to either improve particular sentences (this seems generic, be more specific) and to sharpen the quality of writing (this paragraph doesn’t have a strong topic sentence).

I am drawing upon the ILearnNH guidelines from last class, and on Graham’s advice to try and carve up the writing process into smaller units.

I have thoughts on this. [See this: https://prompts.chat/]

Today’s Plan:

• Drafting a Methodology Section
• For Next Session

Drafting Your Methodology Section

As an English major, I really can’t remember ever having to draft a methodology section. I didn’t do it as an undergrad, nor did I write one as an MA student. I think the first time I ever wrote one was during my PhD, in a qualitative methods research course.

Since then, I have written many of them–both for scholarly pursuits and as part of grant-research and/or assessment teams. Learning to write comprehensive yet concise methodologies is a valuable skill that will benefit you in a wide number of writing jobs. Put simply: before anyone gives you money to do anything, they are going to want detailed instructions on how you plan to spend that money: be it research, web design, grant writing.

Also, as a professional writer, it is not uncommon for you to enter into a project “mid-stream.” Projects will often have begun before you were a part of the team. You might be hired by a pharmaceutical company to write a progress report for a drug trial that started years before you were hired. Or you might be hired by a non-profit to secure a grant and expand an already existing program. In either case, you will likely be handed a pile of material and some data. You will likely have to work to create more data. So, I hope you see how this project prepares you to do that kind of work.

So, the goal for Tuesday is to draft a methodology section that comes in between 500 and 600 words.

A methodology section has a few key expectations:

• First, it needs to detail how you collected your samples/objects/texts (whatever things you analyzed). We analyzed job ads. We collected them. There were criteria for what kinds of jobs we collected. We did not specify a single site for collection. Can you invent a sophisticated yet non-chalant reason for why we were so sloppy? Maybe two sentences on that last point. Make it sound like we are smart and not like we don’t (which, um, means I don’t) have our/my shit together).
• Second, it needs to detail how you analyzed your data. What framework did we use? Does it have a history? Were changes made to the analytical framework in media res? Hint: yes.
• Third, it needs to detail how you ensured/checked the quality/integrity of your data.

Okay, now it is time for a little curve ball. Below I am going to give you some preliminary material for drafting not only the methodology section, but also the report. Some of it is intentionally misleading. Some of it is intentionally inaccurate. I am checking your attention to detail. I am also checking on your ability to distinguish critically important details from minor and less significant ones.

2023 Introduction to the Job Report Project

Our first major project acts as a follow-up to Brumberger and Lauer’s article 2014 article “The Evolution of Technical Communication: An Analysis of Industry Job Postings.” I will ask you to replicate their research methodology in small scale. I myself did this work when I was hired by UNC back in the summer of 2017. I was charged with developing 3 courses that would help both Writing minors and English majors be better prepared for the job market. This charge led me to research job advertisements for English majors, and, at the time, Brumberger and Lauer was the most recent and comprehensive study I found.

However, their article focuses on “technical communication.” This designation can have many meanings–sometimes it is merely a synonym for professional writing. But not in their case–they use it (as do I) in the more precise sense of developing documentation (instruction manuals), product testing (usability reports), and working with scientific experts to communicate scientific/technical knowledge. Their research speaks more to folks at large research institutions with Professional and Technical Writing major, more specialized faculty, and software licenses such as MadCap Flare or Adobe RoboHelp.Our department didn’t have anyone matching those specializations–so as much as I appreciate their research, I wanted something a bit more relevant to our department. We are a much smaller department. While we currently have 7 full-time faculty (myself, Dr. Wood, Dr. Austin, Dr. Ezzaher, Dr. Goodwin, Dr. Brownlee, and Dr. Golson) none of us, I think, would claim Professional or Technical writing as a core specialization–we have experts in Rhetorical Theory, Public Rhetorics, Cultural Rhetorics, Composition and Writing Studies, Rhetoric and Technology. So the question that drove my own research, which you will learn to recreate, is: what skills, technologies, characteristics can UNC focus on to maximize your preparation for today’s job market? How can we best tap into the specializations of our faculty to design both an intellectually rigorous and vocationally strategic program?

In answering that question, I’ve turned my attention to Professional Writing jobs outside of technical writing. During my research, I came across a specialized job listing site–mediabistro.com. From their “About Us” page:

Mediabistro is the premier media job listings site and career destination for savvy media professionals. Whether you’re searching for new job opportunities, striving to advance your career, or looking to learn new skills and develop valuable expertise, we are here to strengthen and support your professional journey. We have the tools and resources to help you navigate your own path and find career happiness.

In addition to job postings, mediabistro.com offers resume services and courses on professionalization and personal brand building. Rather than turning to a more popular site like monster.com, I used mediabistro.com because it focuses specifically on jobs involving writing and communication. I particularly valued it over, say, Indeed.com or Linkedin because of its specific emphasis on creativity. Many of the students taking this course would be English Literature majors; I wanted to do what I could to make sure the course surveyed jobs that a) they might want and b) for which they would feel more qualified.

I spent the month of June 2017 scanning every job ad posted to mediabistro.com. I filtered out jobs that:

• Called for experience in television production (especially those that required years of on-air experience)
• Called for extensive experience as a field journalist (although I retained jobs open to those without journalistic experience; a few jobs were looking for bloggers or content contributers)
• Required degrees in finance or accounting
• Required extensive experience with Google Ads and/or other Customer Relationship Management (CRM) softwares (Salesforce was particularly popular)
• Required applicants bring a client log with them
• Required management or hiring experience (the term management is quite slippery in adverts; sometimes it means “manage a team” and clearly indicates the need for leadership experience. Sometimes it means “manage our twitter account” and isn’t, per se, a leadership position)
• Required backend coding skills
• Required extensive graphic design portfolios (I did retain entry level graphic design jobs)
• Required 5 or more years of experience
• Telemarketing jobs, part-time jobs, or unpaid internships

After filtering out these jobs, I was left with a corpus of 375 jobs. After closer inspection of every add in the corpus, I coded 232 jobs. The data from this coding process can be found here.

I ran this project again in the spring of 2022. My research assistant, Jacob Rigsby, collected another 258 jobs from mediabistro.com. After closer inspection, we culled the corpus to 240 jobs. Jacob first coded 25 jobs from the 2017 corpus for norming purposes. After that practice run, Jacob and I coded every job in the 2022 corpus, meeting on Thursdays to compare results and discuss and rectify non-congruent codes. We made a few slight adjustments to the coding scheme. In particular, we changed “critical thinking” to “analytic and critical thinking” and stopped looking for Cascading Style Sheets and HTML, instead compressing both into one code.

When I ran this experiment for the first time in 2018, job ads were coded by groups of students in my ENG 201 Writing as a Job class. Those codes were compared to my own codes to produce a final coding document.

This document contains final results for both projects.

Today’s Plan:

• Writing?
• How to Read an Academic Article

How to Read an Academic Article

Last class I went over how I read for engagement, moving away from casual reading (just gaining information or enjoying a story) to critical reading (not in the sense of adversity or disapproval, but in the sense of learning how the sausage got made). I provided a heuristic, a set of questions, to help us through a critically engaged reading.

What is Peer-Review?
This weekend I’ll ask you to use this process to read a peer-reviewed, academic article. Academic articles are published in specialized journals. Everything published in these articles goes through a (n agonizing and miserable) review process to ensure quality. First you submit an article, then a desk editor decides if it aligns with the journals priorities. If it does, then they assign 3 professors from around the country to “blind” read and review the article. Those three professors provide extensive feedback and a recommendation that the article be 1) published after revision, 2) revised and resubmitted, or 3) rejected. (At this point, I have published about 17 articles–four were immediately approved, thirteen were R&R’d, and one got rejected. Still mad about it). The process can take anywhere from two months to two years to go from submission to publication. This is why your faculty *really* care about you using “peer-reviewed sources” because those words are imbued with stress, agony, blood, and triumph.

Why Can Academic Articles Feel Impenetrable?
Okay. I think I mentioned this last class–academic articles are difficult to read for a couple of reasons. The biggest is that they are not public-facing, let alone student-facing. Think about the review process I just described: Who is the audience for these pieces? In practice it is for those 3 blind readers who will determine the article’s fate. Pieces are aimed at other professors in the field. Writers feel pressured to make them as sophisticated as possible, to impress the judges so to speak. The language level is, um, “high” (or even “obnoxious” and I admit I have certainly written my fair share of discourse-laden-obnoxious academic stuff).

What compounds this though is that the discourse (specialized vocabulary) of any field–biology, criminal justice, business, history, philosophy, etc–is incredibly complex because we humans have spent a few hundred (or in my field’s case a few thousand) years writing about it. Words come to have special meaning. For instance, if you are reading humanities work on artificial intelligence, you might come to the word “affordance.” Something like “The possibilities given by hypertext structures are one of the most useful and defining affordances of digital technology. However, affordances that are strengths in one domain may be weaknesses in another, and hyperlinks very often distract continuous and concentrated longform reading.” You can probably infer the meaning of “affordance” from context, but you probably cannot infer that the term has a fraught history of debate. Do technologies afford us new possibilities for thinking? Or do the determine the ways that we think? That word, affordances, is caught up in a decades long academic debate. It has a history (and some of those three blind reviewers might be on team “determines” and be pretty angry if you use “afford” and thus you have to construct a sentence that acknoweldges the possibility of determination while not foreclosing on the idea of mere affordance). Sentences get long. Words carry specific academic meanings. Words have histories and allies and enemies.

The other reason reading academic writing is hard is that most academics have not taken a class in writing. They don’t practice writing. And they write passive sentences. This semester we are going to work a lot on writing active sentences. I think active sentences make for better reading because they have a specific character doing a specific action. Take the sentence I quoted above:

“The possibilities given by hypertext structures are one of the most useful and defining affordances of digital technology. However, affordances that are strengths in one domain may be weaknesses in another, and hyperlinks very often distract continuous and concentrated longform reading.”

I believe this is a bad sentence because it has no agent doing something. As a reader, I have to work hard to figure out who thinks or does what. BOO!
Here’s how I would revise that sentence to make it more active: Some scholars believe hypertext will be useful. They think that it will help us to see connections between texts and ideas that we wouldn’t have been able to see before we could link texts together. Other scholars acknowledge that might be true; but they are scared that a constant barrage of links will distract readers and diminish their capacity for longform reading.

Granted, I made it longer. And I used a high-falooting semi-colon. But I also believe I made it more intelligible. How? I made sure every clause started with an agent doing something:

• scholars believe
• They think
• us to see
• we wouldn’t have been able to see
• scholars acknowledge
• they are scared
• distract readers

It is way easier to tell who is doing what in my version because my sentence syntax (the way I treat nouns, verbs, and objects) forefronts agents acting. Academic writing is often agent-less.

Finally, a lot of academic writers do not use “I” or “we” in their papers. This makes it WAY more difficult for novices like yourselves to figure out what is going on. There’s a deep, complicated philosophical reason why some bad writing teachers might have told you not to use “I” in your paper but I do not have time to deconstruct Modern (as opposed to Postmodern) epistemology (the perverse fetishization of Objectivity) and the subsequent demonizing of subjectivity it produced. Just use I in your paper like a normal thinking person aware of their own limitation should. [FOOTNOTE: if you use a weak-ass I in the nature of “”I” indicates this is only an opinion and everyone is entitled to an opinion and an opinion can’t be wrong” then I will swoop in and destroy you. There is only opinion and the quality of an opinion is based on the evidence and reason upon which it is built and I don’t have time to finish writing this paragraph take ENG 319 Rhetorical Theory in the spring if you care about the intersections of identity, power, language, and difference. Take ENG 231 Ways of Analyzing Video Games if you care about identity, power, language, difference, and video games].

Why Undergraduate Students in a Research Writing Class are Sometimes REALLY Bad at Reading an Academic Article
Okay, I’ve been writing for about 30 minutes now, and I think I have covered why academic writing presents difficulties. There’s another one, but it is tied to how y’all do (or, um, don’t) read. And it leads into my first big tip for productively engaging academic writing. Let me take a sample article and walk you through. It is so poorly written that I would fail it in this class. For reasons I will make clear to you now and consistently over the next few months.

To help facilitate this read, let me first give you the expected structure of an academic article (note: not every field will use this exact structure; literature, history, and philosophy do NOT do this; nursing, critical justice, education, and any science almost certainly will).

• Abstract
• Introduction
• Literature Review / Background Literature
• Methodology / Methods
• Data / Findings
• Discussion
• Conclusion

Okay, now let me annotate what those sections are supposed to do [Ideal] and what they often end up doing [Real]

• Abstract: this should be a 300-400 word summary of the project. A good abstract will indicate specific major findings. A bad abstract will not.
• Introduction: this should essentially be a slightly longer abstract. The opening paragraphs might make a case for the importance of the research. One paragraph should gloss (quickly summarize) the methods section. One paragraph should detail every major finding. If you read an introduction and it leaves you questioning what their experiment actually found, then you are reading bad writing. Academic writing is not an episode of Scooby-Doo. Do not pull the mask off the findings on page 15. Tell me on page one. I will fucking die on this hill.
• Literature Review: This is an essential part of the peer-review process. It demonstrates to reviewers your awareness of the field. How? By making you summarize and link all previous relevant research in the field of discussion. It is like saying “see, I’ve read all the things I am supposed to have read to write about this topic.” STUDENTS: you are generally bad at reading academic research because you spend a lot of time reading *this* section. Which makes a ton of sense–you are all new to basically every field and if you are interested in a field then you should spend time here learning its foundations. AND–please don’t take this personally–research shows that you just don’t spend a lot of time reading for homework and so are likely to burn out after about 15 minutes of reading (which, um, isn’t enough) but if you are only going to spend 15 minutes reading an academic article then for the love of the gods do not spend those 15 minutes reading the lit review
• Methods: this should explain three things:
  • How the researchers collected the people / objects they were going to study. As readers we should be interrogating these methods for bias [sample size, etc]
  • How the researchers analyzed the people / objects they collected. Ways of analysis differ *really* widely between different fields. From ways of reading to some serious mathematical stuff that I do not understand at all
  • How the researchers made sure their analysis was reliable and valid
• Findings: Some disciplines connect findings and discussion into one section. Others do it separately. Findings sections can involve some impenetrable math. Or tables that will not make sense to a non-expert.
• Discussion: HERE IS THE MONEY. This is where the researchers clarify findings and discuss their significance. Ideally, they will have summarized discussion findings in the introduction. But about 75% of the time here is the stuff we are looking for.
• Conclusion: usually phatic nonsense about how this one study is a start in a direction but ultimately more research will be needed to blah blah blah.

Okay, let’s look at a better article.

For Next Session

Guess what? Y’all are going to read an academic article. I did not have time to read all of these, but I have scanned them and believe they should be 1) interesting and 2) accessible.

• Creativity: Haase and Hanel.
• Racial Bias and Education: Warr-et-al.
• Pedagogic Potential (Education): Sandu-et-al.
• Regulation: Rao AND Lebouhk (two short pieces)
• Economic Impact: Noy and Zhang.

I want you to read an academic article and then write about 500 words about it (2 1/2 pages double-spaced). Half of that writing should try to summarize the article. Try to briefly summarize the methods and focus on the findings. But then get to the important stuff: ask yourself, “what does this article want [folks] to know?” and “with that knowledge, what does it want [folks] to do differently?” What does it want readers to do? What does it want other people to do? That’s the summary.

Then, do some thinking. We’ve read a bunch of Mollick. You have read other public pieces on AI. You’ve been using AI. Think. Connect. Contrast. Question. Write. Do you have questions about their methods? Do you agree with their finding? What questions do you have? Do they mention something else that you should read? There’s no one right way to think. And I don’t want to try and predict or limit the vectors you might pursue. So I simply ask for a word count (which you are welcome to blow past) as some sign of engagement. Gut responses and then reflection.

As always, you are free to use AI to help you with any element of this assignment. Just tell me how you are doing that. This can be integrated into your writing, or can be a sentence at the beginning to alert me to what’s going on.

Today’s Plan:

• IFS Welcome to the Space Show / Linduo Auditorium / 7pm
• Engaging An Article (How I Read Research)
• For Next Session (in the Ross Computer Lab

Engaging an Article (How I Read Stuff for School)

I’m not trying to be condescending here–I know y’all know how to read. But I do want to cover how to engage a reading and highlight the specific things I do when I’m reading material that I think I’ll end up writing about.

In rhetoric we call these things “heuristics,” a set of loose procedures to help read, write, or think. The one I generally teach in this class is called a “worknet.” The worknet approach was developed by Derek Mueller, a writing professor at Virginia Tech, in his 2015 article “Mapping the Resourcefulness of Sources: A Worknet Pedagogy. Mueller’s process has four phases; I want to talk about three of them:

1. Semantic
2. Bibliographic
3. Affinity

The Semantic phases focuses on vocabulary. I give a reading a quick gloss looking for important key words, phrases I don’t recognize, buzz words. What words do I need to investigate further? What words help me position this text in a discursive field, context, on-going conversation?

The Bibliographic phase looks at a writer’s sources. But to do this, we don’t turn to a works cited or reference list (assuming the piece even has that). Instead, we want to look for whomever gets cited or referenced in the work. Quotes or paraphrases. And then we want to focus on the ones that are important to the argument of the whole piece, not a drive-by or hook. What are the voices (sources) with which the writer wants to align themselves?

The Affinity phase looks at the writer themselves–where did they go to school? Where do they work? What else have they written? With whom do they write? What attitudes or agendas (and I mean that as neutrally as possible) do they bring to their writing?

When I am reading something–academic or otherwise–I’m not only thinking through Mueller’s worknet, but am working with a few other things in mind.

• First, I am looking for how the writer treats their opposition (assuming I am reading an argumentative thing). Do they work to fairly present opposing view points? Do they cite sources or rely on generalizations? Do they treat opponents evidence fairly or just dismiss it? Counterargument is central to quality thinking and democratic practice. How we treat words equates to how we treat people. I want to read and think with people who are nice.
• Second, I am looking for how the writer acknowledges (or doesn’t) the limitations of their own position. In academia, we call this “qualifying” claims. There’s a pretty significant difference between writing “AI will end education as we know it” and “AI presents a threat to the way we have thought about school” and “AI will require that students, parents, teachers, and administrators thoughtfully consider how to integrate into our schools.” The first one has a certainty to it that irks me.
• Third, and perhaps most important, I am always asking myself what the author wants me to get from the thing I am reading. What is the purpose? What do they want me to do differently?
• Fourth, I am thinking about the rhetorical relationship the author is asking me to accept, the identity they want me to adopt. Who do they want me to be? What do they expect me to think?

Not every one of these questions will be relevant to every reading I do. But taken together, they provide me a wide range of strategies for engaging a reading and positioning it on a map with other readings. They provide a path to thinking. So, that list:

• Special Vocabulary
• Important Sources / Evidence
• Who the writer is, what they do, (and for academic stuff) what else they have written, where they went to school, with whom they write
• How they treat their opponents
• How sincerely they engage counterarguments
• How they qualify their arguments
• What is the central arguments? What does the writer want me to do differently (idea > action)
• How the writer sets up readerly identities

As I read, I can try and figure out how to start writing. Do I have to research a term? Look more into sources? Imagine a stronger counter argument?

For Next Session

Here’s the potential reading list:

• “Why AI Isn’t Going to Make Art.” Chiang. AND “AI-Generated Art Won a Prize. Artists Aren’t Happy.” Roose. [On Creativity]
• “How AI could explode the economy” AND “Is AI Overrated?”
  [Economy / Regulations]
• “Why I’m Banning Student AI Use this Year” Boryga. AND “How AI Could Save (Not Destroy) Education.” Kahn. [Education]
• “How artificial intelligence can help combat systemic racism.” Murray AND “Colorado law enforcement welcomes speedy AI facial recognition technology along with rules, some advocates worry about privacy and misuse”. Sherry. [Alignment / Bias]