Categorizing acceptability judgments

Goss, Zhang, & Lantolf (1994) were actually the first to use a discourse analysis approach to AJTs. Here's what they say about characterizing the participants' judgments (their tasks were done in groups):

"...we need some criterion to differentiate between talk that reveals evidence of judgments based on genuine linguistic intuitions and talk that points to some other source, such as memorized rules or L1 translations, underlying the judgments."

[Personally I don't care especially about that difference at this point, but I do want some kind of taxonomy along these lines.]

"At the moment we believe that talk that cites rules, pedagogical or otherwise and accurate or not, the source of which may be an instructor or textbok, or that involves translations into the native language, is indicative of judgments based on something other than real linguistic intuitions."

[Again, leaving aside the question of determining 'real linguistic intuitions,' this starts to suggest a way to categorize.]

"On the other hand, talk characterized by the absence of rules, by remarks like It sounds right, or a difficulty on the part of respondents in putting what they know into words, we construe as evidence that abstract linguistic competence of some type underlies the judgments."

[This is good stuff for the interviews, definitely, when participants are asked to explain their judgments.]

Higgins (2003) and Rubdy et al (2007 & 2008) look for evidence of ownership:

"...not just indigenization in which speaker appropriate English for their own needs but the degree to which speakers 'project themselves as legitimate speakers with authority over the language.' (Higgins)

Higgins uses some ideas from Erving Goffman, of all people, to get at that. Between the three studies there are 5 things they look for in the data:

1. references of legitimization (appeal to external authority)

2. Use of pronouns ('you' or 'I' (or we?) rather than 'it's wrong)

3. modality indicators (what you can, can't, should use)

4. References to the speaker's own English usage 

5. References to intuition / not being able to explain

I might add

6. Teacher-y comments about incorrectness/error and/or direct suggestions for changes

So, all together now, based on this and what I recollect from the data I've seen in my own study so far

1. appeal to external authority

a. textbook/dictionary

b. teacher/how I was taught

c. native speaker or other expert speaker usage (eg 'Americans don't say this' or 'I've never seen this' - although maybe this could be under #4 also)
d. reference to grammar? Does a reference to grammar automatically mean 'external authority?"

2. Use of pronouns

a. "You say" (indexes authority - I tell you what to say)

b. "We say" (indexes membership in community of expert English-knowers)

c. "I say" (asserts individual authority as an expert English-knower)

d. "They say" (if referring to a preferred usage, a reference to expert/NS use; if referring to a dispreferred usage, a reference to learners or people who don't know English well)

3. modality (can, should, may, etc -- could analyse based on 'strength?') + how it is modified (definitely, etc) - not quite sure how to classify these.

4. References to the speakers' own English usage/knowledge (broaden, include S usage as in 2d?)

a. "I would say" (similar to 2c or 3, I suppose)

b. References to what is taught (I/we teach them to do X)

c. probably some other stuff...?

5. References to intuition - more for the interview, but could be things like:
a. feels, seems, etc (suggests strong intuitive sense of language?)
b. Vague/ambiguous adjectives ('this seems weird')

6. Teacher stuff
a. Simple "this is wrong" stuff. (also 'this is Chinglish') - could specifically mention an error
b. correction without mentioning anything else (could include modals)

...to be developed...