Posted by Daniel Lerner in Discussion: Putting a meaning to the number (Graduate Evals)

Posted by Nathan Miller in Discussion: Putting a meaning to the number (Graduate Evals)

Posted by Captain J Ridgley (Commandant) in Discussion: Putting a meaning to the number (Graduate Evals)
Posted by… suppressed (1) by the Post Ghost! 👻
I think it would be extremely helpful for anyone looking at participating in this conversation to see the current standards for the scores. None of these can be scored a 0 - scoring a 0 in any of these areas would essentially mean we shouldn’t be evaluating this cadet for graduation at this time - this is pretty clear from the 0 score definitions you used below. Why would we evaluate someone who never posted? Who had no sim interaction, or who can’t read or write? In addition, it is important to remember that this is not a writing academy, but an academy to introduce new players to our game. The goal is to relieve the burden from the main fleet to teach these things, not to output world class writers.

AWOL Limits
1 - Was frequently AWOL and/or did not declare an LOA properly most times
2 - Was sometimes AWOL and/or did not declare an LOA properly some times
3 - Posts within AWOL limits and can demonstrate how to declare an LOA
4 - Often posts more often than the AWOL limit and can demonstrate how to declare an LOA
5 - Frequently posts more often than the AWOL limit and can demonstrate how to declare an LOA

Writing Style
1 - Posts are not legible and have mistakes in at least three areas listed below
2 - Below average use of our writing style. Posts are not easily legible and have mistakes in two or three areas listed below
3 - Average use of our writing style. Posts are easily legible, but may have mistakes in ore or two areas listed below
4 - Above average use of our writing style with mistakes in only one area listed below
5 - Excellent writing style with little or no mistakes

Understanding of our writing style: correct spelling, punctuation, grammar, and writing in the past tense.

Post Presentation
1 - Does not adequately demonstrate three or more areas of post presentation listed below.
2 - Does not adequately demonstrate one or two areas of post presentation listed below.
3 - Adequately demonstrates all areas of post presentation listed below but does not excel in at least three areas.
4 - Excels in demonstrating at least three areas of post presentation listed below and adequately demonstrates all areas of post presentation.
5 - Excels in demonstrating all areas of post presentation listed below.

Understanding of post presentation, consisting of five main areas: Trimming, Signature, Location headers, Paragraph form, Speech symbols (=^=, “”, **, etc.). - Someone who excels will always do all of these things properly. - Adequate demonstration means that the Cadet knows how to do the items listed but may sometimes forget. - Inadequate demonstration means that the Cadet has not shown the ability to do the items listed.

Sim Interaction
1 - Does not participate as an active crew member
2 - Only participates when prompted
3 - Manages to be in the main sim and participate acceptably without prompting (little or no interaction in side sims and is not a driving force in the main sim).
4 - Manages to be in side sims to keep busy, or actively participates in main sim (but is not a driving force).
5 - Creates side sims to encourage others to interact, or is an active force in the progression of the main sim.

Timeline Management
1 - Is not able to be in more than one timeline at a time
2 - Struggles when acting in more than one timeline
3 - Manages to be in more than one timeline with little or no issues
4 - Manages to be in more than two timelines without issue
5 - Manages to be in more than three timelines without issue

Character Limitations
1 - Character is unrealistic for the Star Trek universe
2 - Character has “super-character” tendencies, or is otherwise somewhat unreasonable but is still acceptable
3 - Character is reasonably conceived and played
4 - Character has more than average depth, and is played reasonably
5 - Character is extremely well thought out, with flaws, history, personality, and strengths within a reasonable limitation

Crew Interactions
1 - Does not interact well with crew and/or frequently demonstrates selfish playing tendencies.
2 - Has trouble interacting positively with crew and/or demonstrates selfish playing tendencies.
3 - Interacts well with other crew and is a team player but does not initiate interaction.
4 - Initiates positive crew interaction OR demonstrates being a team player.
5 - Initiates positive crew interaction and demonstrates being a team player frequently.

---

I find the recommendations below to be too stringent in areas where flexibility is required, and too relaxed and vague in areas where specificity is needed.

Posted by… suppressed (1) by the Post Ghost! 👻
Hello, one and all!

So this month will be one full of discussion and I am completely open to criticism and rebuke. This is to simply achieve a clear picture and betterment of the Academy process. With that said, in the background, I have been in discussion with how we grade the cadet Evals at the end of their term. As such I noticed a few differences between grading and recommendations for Cadets in regards to their role and their grade, subjective was the word that came to mind. So I have put together a very draft based idea of a ‘meaning to the number’ so that 1-We have a clear understanding of what a score means to the Cadet and 2 for me most importantly we are then able to go back to the Cadet with work on points and coach them into being a better writer.

This scoring system has been adapted from my own personal experience working in a military training establishment.

So here it is, (We already grade in the following subjects if you wasn’t aware and is typically out of 35.)

(Only a recommendation, this is by no means resolute.)
28-35 = DH
19-27 = Main
11-18 = Main (At-Risk could require a mentor)
0 - 10 = Improvement Required

This currently lowers the expected scores for graduation to the main fleet - a score of 21 is graduated to the main fleet; below 21 may be graduated with explanation, or may be recommended for remediation. A score of 21 is someone who is scored a 3 (adequate) in all 7 areas, or is very high in some areas and low in other areas. Someone who scores an 11 could be below average in all categories; someone scoring as high as 14 could be below average in all categories. Heck, someone who is never AWOL but scores a 1 in all other categories would get an 11. Even with a mentor after graduation, is that someone we should be graduating?

AWOL Limits:
0 - Has Never Posted
1 - Has spent 75% time AWOL or for a duration of over two months.
2 - Has spent 50% time AWOL or for a duration greater than a month but less than two months.
3 - Has spent 25% time AWOL or for a duration greater than a week but less than a month.
4 - Has infrequently been AWOL or for a Duration less than a week.
5 - No AWOLS.

Note that this completely excludes the LOA setup currently required; and shifts to decidedly more lax policy on AWOL. Currently a 3 is no (or few, with none by the end of the eval period) AWOLs; in this scale, someone could miss a huge chunk of a sim and still get a 3 (considered adequate).

Writing Style:
0 - Writer shows no understanding of STF writing styles
1 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC notes.
2 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, 1st and 3rd person views.
3 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, 1st and 3rd person views, Speech and Actions.
4 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, 1st and 3rd person views, Speech, actions and thoughts.
5 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, 1st and 3rd person views, Speech, actions, thoughts and descriptive writing.

The thing I don’t like about this is the concrete assumption that they must do A before B. What if someone shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, Speech, actions, thoughts and descriptive writing, but doesn’t demonstrate knowing the difference between 1st and 3rd person views? I have no way to score them. Or demonstrates descriptive writing but never clearly separates IC and OOC? The current system allows for combinations of the required areas.

There is also no definition of what “basic STF writing styles” means. Currently, what is covered by the Writing Style category is defined: “correct spelling, punctuation, grammar, and writing in the past tense.” These changes seem to blur the line between the Writing Style and Post Presentation categories. Writing Style mostly covers anything non-STF related, while Post Presentation mostly covers those things uniquely STF.

Post Presentation:
0 - Writer is unable to clearly present their posts, due to combined missing use of paragraphs, grammar and spelling. (Can not read nor write)
1 - Writer shows understanding in the use of grammar.
2 - Writer shows understanding in the use of grammar, spelling.
3 - Writer shows understanding in the use of grammar, spelling, paragraphs.
4 - Writer shows understanding in the use of grammar, spelling, paragraphs, IC and OOC notes.
5 - Writer shows understanding in the use of grammar, spelling, paragraphs, IC and OOC notes and signs off posts.

See above for the inflexibility of this system and the blurring of Writing Style and Post Presentation. Between the two, we seem to have dropped trimming posts, dealing with Location headers/post titles, and speech symbols (which may also include a basic Markdown tutorial for italics, bold, etc).

Sim Interaction:
0 - No Sim interaction.
1 - The writer infrequently participates in either the main or side Sim interaction.
2 - The writer infrequently participates in both main and side sim interaction.
3 - The writer frequently participates in either main or side Sim interaction.
4 - The writer frequently participates in both main and side sim interaction.
5 - The writer helps lead either the main sim or a side sim interaction.

This is vague, as it doesn’t define what frequently and infrequently mean. The current requirements better define the activity level at each point, and have a larger focus on participation in the main sim.

Timeline Management:
0 - No Timeline Management.
1 - The writer infrequently participates in one single timeline.
2 - The writer frequently participates in one single timeline.
3 - The writer infrequently participates in two or more timelines.
4 - The writer frequently participates in two or more timelines.
5 - The writer helps leads one timeline while participating in multiple timelines.

I don’t feel like “leading a timeline” is appropriate for this category. That falls under the Sim Interaction category, which already mentions leading a thread.

Character Limitations:
0 - Character has no limitations, has beaten the kobayashi maru without cheating. OR to many.
1 - Character shows very few limitations or far too many.
2 - Character shows very few limitations and the writer understands the importance of avoiding god complex.
3 - Character shows a healthy balance of limitations
4 - Character shows a healthy balance of limitations and the writer understand the importance of avoiding god complex.
5 - Character shows a healthy understanding of limitations as well as the understanding of avoiding god complex and has shown character development during their evaluation period.

“Character development” usually means a growth or change in a character, which isn’t always accomplishable in the short period of time a player is in the Academy. The current system better captures what we’re looking for and what we’ll have time to see in the Academy: “Character is extremely well thought out, with flaws, history, personality, and strengths within a reasonable limitation.” In comparison to my comments earlier, I feel like this is a category where an expectation of A before B is acceptable, so not having “no god complex” in the 3 point tier is strange.

Crew Interaction:
0 - No crew interaction.
1 - The writer infrequently interacts with at least 10% or less
2 - The writer frequently interacts with at least 10% or less
3 - The writer infrequently interacts with at least 50% of the crew.
4 - The writer frequently interacts with at least 50% of the crew.
5 - The Writer infrequently interacts with at least more than 50% of the crew and frequently with their department.

These numbers are… practically unachievable, imo. Currently there are 12 crew members on the Challenger. To get a 3, I’d have to interact at least once (I think - how often is “infrequently”?) with 6 people. A full department normally is 2 cadets; that means in a main sim thread in my department I’ll likely interact with 2 people - my DH and the other cadet. In a side sim, I might be lucky to get 2 other people participating. There simply isn’t time to do this in the time frame required. A cadet will be in the Academy 3-4 months usually; this means they may only end up posting 12-16 times, and still be within AWOL limits; heck, under these guidelines they could end posting only 9 - 12 times if they’re AWOL 25% of the time.

This is also an area where currently we look at the quality of the interaction, and not the quantity. Do they take over threads? Do they ignore other players? Do they always try to steal the spotlight? If we’re not looking at quality, the rest of these categories are for naught and we might as well just be looking at post counts.

---

I find the current scoring system to be better defined, and better at determining whether we’ve accomplished the goal of preparing a cadet to enter the main fleet.

As an example of a player that would graduate under the current system with a score of 3 in each category, someone must:

• Post within AWOL limits and know how to declare LOA when they need more time
• Make legible posts with minor errors in spelling, punctuation, grammar, or writing in the past tense
• Know how to trim posts, sign posts, use correct location headers, appropriate paragraph usage, and appropriate speech symbol usage, but may occasionally forget
• Participate in the main sim without prompting
• Be in more than one timeline
• Create a reasonably conceived and played character
• Interact positively with crew and be a team player

An example of a player that would graduate under the recommended system, with the same score as the above:

• Is AWOL 25% of the time (3 - Has spent 25% time AWOL or for a duration greater than a week but less than a month.)
• Writes in the present tense (3 - Writer shows an understanding of the basic STF writing styles and knows the difference between IC and OOC posts, 1st and 3rd person views, Speech and Actions. note: there is no mention of tense)
• Does not trim or sign posts (3 - Writer shows understanding in the use of grammar, spelling, paragraphs. note: there is no mention of trimming or signing posts)
• Only participates in side sims (3 - The writer frequently participates in either main or side Sim interaction.)
• Plays in mostly one timeline (3 - The writer infrequently participates in two or more timelines.)
• God modes (3 - Character shows a healthy balance of limitations note: scoring a 2 or a 4 in this category mention god complex characters, but 3 does not)
• Takes over threads with half the crew (3 - The writer infrequently interacts with at least 50% of the crew. note: no mention of what types of interactions are acceptable)

As someone who has done evaluations in the past, and will be shortly doing more, based on the information provided in this post I would have to recommend this player for graduation, either because I was forced to by the rule itself or because of a lack of other detailed requirements (Writing Style, Post Presentation, Character Limitations, and Crew Interactions in particular).

Some of these are relatively minor, really - but which one is better? To me, it is clearly the graduate under the current system.

Sincerely,

Nathan Miller
Challenger CNS

Wow!

Thanks for the feedback Nathan, I really appreciate it. I had asked about for the current scoring system for the past months or so and couldn’t find where this was and thus resulted in me making a new one, frankly I agree with you. The current scoring system make more sense.

As a result I’ll have a full look at the scoring system that is currently in place, but I’m already happy with what you’ve presented. Perhaps I didn’t ask enough people.

Can you please share any further details you may have in regards to your scoring methods. As it is my hope to have this added to the form.

Kindest regards,

Jake

Hey Jake,

This was all taken directly from the current form.

As for my scoring methods? Do you mean like the above, or my personal evaluation methods?

Nathan

The current system is meant to be an objective scoring system. “If Cadet A has done all 4 of the following, score X. If Cadet A has done 3 out of the following 4, score Y.” It’s a bit more nuanced than that - it sometimes separates “done it but when prompted” verses” “does so on their own initiative”, and things like that - you’ll still get slghtly different scores for the three evaluators, but they shouldn’t be too far apart. When they are far apart, there can be a discussion about that.

Daniel

Given that there is indeed a pre-exsisting number scheme I think its safe to say this topic is resolved?

Feel free to add any comments, or even ideas.

Jake