First, the good news.
1) My colleagues and I were generally in agreement about how to grade student work. I was worried that we would have wildly different opinions about what constituted a 2 vs a 3, but it turns out we are in agreement most of the time!
2) I like the rubric, generally. I think the explanations of each rung on the ladder of mastery make sense to me. I might have some quibbles with the conversion to percentages or the Level 5 standard, though. In my view, Level 5 should be something like "You can apply the concepts of this standard to new and unfamiliar problem types" or something like that.
3) It didn't take me much longer than usual to grade the tests. Since I hand-grade every test and only include open-response questions on my quizzes, grading takes me a while anyway. So this didn't add much time to my grading. My colleagues to somewhat longer because they didn't write the quiz so they had to work or re-work the problems to check student answers.
Now, the bad news.
1) Our school requires some form of percentage grading at the end of the year. That means we'll have to convert standard mastery into a number grade at some point. There are a bunch of ways to do this, but they are all predicated on weighting every standard equally. In my opinion, not every standard we teach is equally important to student success or numeracy. So there will always be that tension when writing the standards. Do you break a more important standard into substandards so that its weight is appropriate, given its importance to future success in math? Or do you just write all the standards without this consideration in mind?
2) I tried to write my standards as "I can" statements so that I could easily assess them. Now I feel that this was a mistake. First of all, there was this question on the test:
A line is drawn through the points (5, 8) and (9, 10). List another point that is also on that line.
I always have a question or two like this on my tests - something I never explicitly taught in class, but which can be solved by applying the concepts we've learned in class. This might be a Depth of Knowledge Level 2 question, but baaaarely. Only because I never taught a problem like this in class.
As a result, I don't want this to be a standard in my grading, or at least not an "I can" standard. If I include something like "I can find additional points on a line, given two points or a point and a slope" then I have to teach that explicitly in my class or else it's unfair to include on the test.
My point is, these standards, as I wrote them, are oriented toward answer-getting instead of conceptual understanding. And I worry that much of standards-based grading encourages students toward answer-getting. It's a much more communicative grading system, but it's still about "Can you solve this type of problem? What about this one? What about this one?"
So I tried rewriting my standards as "I understand" statements, such as "I understand the relationship between an equation for a function and its graph"
The best thing I came up with for my challenge problem: "I understand the relationship between the slope of a line and the points along that line."
I am not very happy with that language. And I also think that this sort of "I understand" framework would lead to a lot more disagreement between my colleagues and me.
On the positive side, if I changed everything to "I understand" then I could change that "5" category in the rubric to something closer to "5 - I can apply this concept to unfamiliar and non-routine problems." This would help me reach my overall goal as a teacher, which is to set the standard that 100% indicates more than procedural fluency on routine problems.
3) This process has helped me to see some of the drawbacks of standards-based grading, such as the ugly conversion to percentage grades or the tendency to focus the standards around answer-getting rather than conceptual understanding.
So I emerge from this process more skeptical of standards-based grading. But I am EVEN more skeptical of my current grading system. Yes, there are flaws in any grading system. But every time I encountered a flaw in standards-based grading, I thought "Well, does my current system address that problem any better?" The answer was usually no.
So now I feel obligated to move to an imperfect grading system next year because it's a lot less flawed than my current system. I don't think I can keep grading strictly on a percentage-of-questions-right model after this year. It's too blunt a tool.
Now I just have to make a comprehensive list of standards for both of my classes, find or create DOK level 2 or 3 problems for each of those standards, create a procedure for assessment, remediation and re-assessment, make some form of document or video explaining my system to students and parents, find a computer program that easily logs and communicates each student's progress through the standards, and decide how my school's required semester exams should be incorporated in this system. Piece of cake.