Last year I spent 8 days for my last company CMMI appraisal, one day as a readiness check, then two days as a pre-appraisal training, and 5 days in formal CMMI 1.2 appraisal as an ATM (Appraisal Team Member).
In the readiness check, we reviewed the PIIDs document and other documents. PIIDs is an excel sheet that maps all CMMI Level 2 practices to Direct Evidences and Indirect Evidences from the appraised company.
In the two days pre-appraisal training, we learned many interesting things about the upcoming formal appraisal, whatever I had learnt the experience of the formal appraisal itself is the real experience that no training will substitute. Here are some of my notes and experiences.
Working as a Team: In the appraisal, we should work as team. Most of the time the Lead appraisal is facilitating our discussions and decision. Decisions are taken by consensus. It seems strange, but was helpful to objectively discuss our different opinions and encourage members with different views to explain their point of view and try to conclude one decision.
Goals and Practices: Although (as I learned in Introduction to CMMI 1.2 course) Goals only are mandatory, practices are also important, if you are going to ignore some practices, you should show that you replaced them with other practices that are finally achieve the mandatory goal. In other words, practices are mandatory but can be replaced as long as the goal is met.
Direct Evidence: For each practice, you should show direct evidence document, this means a lot of documentation, it seems this is not replaceable but I am researching ways to reduce the number of documents needed. Reducing documents to the necessary minimum is important as my focus is on small business where I want to add value and minimize overhead.
Balance between Interviews and Documentation: You are obliged to balance between documentation and interviews, you got affirmation from interviews you formally hold with team members. CMMI enforce you a minimum set of affirmations to achieve the balance between documentation and affirmations.
Taking Notes: It was astonishing to know we should take full notes in interviews to remember what was heared and seen. Notes should be accurate as said without paraphrasing. After the interview we should tag the notes with project and Process Area. This helps to see conflicts and get confidence in reporting affirmation.
In the last day we destroyed all our notes per request from our laid appraiser to prevent out of context interpretation to our notes. This makes sense as we already decided about rating the company and finding any mistakes later will not help.
Sub-practices Review: When we feel any doubt about if the evidence is sufficient in a practice, we review the sub-practices and see if there is anything missing.
Overlap and Recursive Applicability: An overlap is exist between process areas, and between generic practices and process areas. There exist some recursive concepts like plan the plan, or put configuration records in the configuration management system or make quality assurance on the quality assurance activities.
One statement I learnt from our lead appraiser that helped me was “Don’t jump to conclusions”. It showed to me that many time we jump to conclusions that we found wrong after getting more information.
At the last day, all team members wrote their feedback to SEI website, we take pictures and was very happy. Yes the company is appraised as CMMI Level 3.
Here are the lessons I have learned from my second CMMI L3 appraisal, not all of them are technical:
1) Let actions speak about you and don’t speak about yourself; doings are more respectable than sayings. Most people hate who speak much about themselves.
2) Dictatorship: If you are a team leader and unable to facilitate team discussions or reach consensus, this usually reflect your dictatorship style or the mentality of win-lose. As a team, everyone must focus on win-win relationship. We should believe we are strong because we are different and being different means we have a great chance of building integrated vision.
3) Treating people should be smooth, not just when things are easy but also with hard issues and when we feel angry. IMHO this is the measure of person kindness and professionalism.
4) You can always ask people for annoying things as long as you treat them with great respect and let them understand your intention. As example, if you in a critical meeting, you can ask a person to leave the room, or to turn off mobiles as long as you say it properly. On the other side, some people feel great when they have the chance to hurt other people.
5) CMMI interpretation is quite flexible, you can tailor it greatly. Remember that CMMI is a model not a process. You can use this feature to hurt your business or to help it.
6) Using tools improperly can lead to a very complex baseline taking and reconstruction. So always use tools the right way.
7) Tools come with their own complexity. Use sophisticated tools only to solve complex problems. Generally use the simplest tool that is good enough to manage your current problem. Use the right tool for the right task.
Here are my tips on practical implementation of CMMI as I experienced in my first appraisal:
1) In CMMI 1.2 PP (Project Planning), you have to create Project Plan, however the GP (Generic Practice) 2.2 require you to “Plan the Process”. It sounds like you have to create a plan to create the project plan. In my experience it is sufficient to include planning tasks in the plan, as example of planning tasks:
– Create detailed WBS (Work Break-Down Structure)
– Determine and analyze risks
– Discover and plan for data management
– Research for needed resource and skills
– Meet to review the plan
– Modify the plan (according to the meeting)
– Final meeting to get plan commitments
The same concept apply on GP 2.8 “Monitor and Control the Process”, it is sufficient to include tracking tasks like milestone review and weekly progress review. These two generic practices gives the organization detailed information about how much planning and tracking cost you in the project.
2) In PP, Specific Practice (SP) 2.3 “Plan for the management of project data”, it was sufficient to put this planning as part of the CM (Configuration Management) Plan.
3) It was a weakness to let the CM baseline creator to audit that baseline. As he will not be objective as he is biased to his creation. It is accepted to let a project team member to make the baseline audit, or make two project managers to review for each other.
4) In PP, SP 2.1 “Establish and maintain the project’s budget and schedule”, It is accepted to have the project effort as the budget!. Why?, the senior manager – as the organization is very small – don’t share the cost details with the project managers. However it is recommended to the organization to document the procedure of converting effort to cost.
5) The PMC SP 1.6 and 1.7 dictate to review periodically and on milestone the project. However it was accepted to align the milestone with the periodic review to be the same.
6) All MA (Measurements and Analysis) planning and tracking activities are stated in the project plan, however the measures will be collected and analyzed in a separate documents. Regarding using tools, we can just use office and file server to do all what you need. Configuration management system is useful when developing source code, if the project is just migration or conversion of legacy source code, then the use of CM system can be dropped.
7) There is no paper based sign required, all the documentation and approvals accepted to be emails or just MOM (Minutes Of Meeting) documents.
8) Even not required in L3, the creation of templates is very helpful as there exist many details that can be forgotten.
9) In PIIDs (Practice Implementation Indicator Descriptions), it was not necessary to write many indirect artifacts, as affirmation through interview can be almost sufficient, except in case it is hard to get that affirmation.
10) You can easily plan the appraisal and meet your plan (without too much overtime) and even have fun.