Are there any objections to making our top level OKRs map to our ToC pillars?

[jawache]

The SC has agreed on a top level set of OKRs and we were starting to have discussions with chairs about working group level OKRs. 

TL;DR: Are there any objections to making our OKRs map to our ToC pillars?

• An Objective is the place we want to get to.
• A key result (KR) is a hint, a signpost, that we are going in the right direction to meet our Objective.
• KRs change all the time, objectives should be pretty static over the years.
• We have a PM who's responsible for capturing, tracking and reporting all the OKRs for the whole Foundation. Top level and project level OKRs.
• We don't have to get OKRs perfect straight away, we're just starting this process so we are going to get things wrong and need to adjust over time. This is our first try.

Our Theory of Change is either central to everything we do or it isn't. If we want it to be central we need processes to make it central, it's not going to happen by accident. So instead of having OKRs per Working Group, I'm proposing we have OKRs focused on our Theory of Change. Our working groups, our projects, all of it should really be in support of one or more of our Theory of Change pillars.

OKRs

OBJ1: Attract and retain members
KR1.1: 80% of our members renew their subscription every year (They invest money).
KR1.2: 80% of our steering member orgs have attended at least one board meeting in the last 3 months (They invest their time)
KR1.3: 20% of our individual members are active every week in GSF. (An active member performs at least one action on GitHub every week)
KR1.4: Maintain revenue for 2023 at 2022 levels (to achieve this we have to attract new members to offset the loss of members due to attrition)

OBJ2: Change tech culture
…so the environmental effects from software are made a priority.
KR2.1: 10K users are members of GSF meetup groups by end of 2023
KR2.2: 10K newsletter subscribers by the end of 2023
KR2.3: Speakers give 100 public green software talks in 2023
KR2.4: 200K people have viewed at least one insight from the SoGS report by end 2023

OBJ3: Change knowledge
…so the environmental effects from software are studied, taught and understood.
KR3.1: 1 million people have enrolled in one of our training courses by the end of 2023
KR3.2: 5 academic institutions reference green software foundation content in their courses by the end of 2023.
KR3.3: We have 50K visitors a month to our own sites by end of 2023

OBJ4: Change tooling
…so the environmental effects from software are easy to act upon..
4.1: 12 SCI case studies published by end of 2023

UPDATE 21.10.23 : Removed old KR3.3 re: number of patterns and replaced with a KR around further/external education as per the discussion bellow with @tmcclell and @Sealjay

UPDATE 24.03.23 : Added KR4.1

[jawache]

@mrchrisadams & @elisezelechowski - Policy WG speaks the most to OBJ2, the only KR here policy relates is the SOGS report. I think that's ok to start slow and build up, but let me know if you want others.

[jawache]

@tmcclell & @aecurrie - Community WG has a lot of KRs here esp. in OBJ2 and OBJ3.

[tmcclell]

With regards to patterns, I think the goal is definitely a stretch goal as we have 40 now? Also, I would like to see OKR's around carbon goals since our mission is to reduce carbon emissions through applications and OBJ4 does start to address that, and it may be sufficient for 2023 to start.

[jawache]

Ok I can remove 3.2, from the CommWG call sounds like this is a real stretch goal, we can look to add it in again later once things settle down.

Will add 3.4.

@tmcclell w.r.t. carbon reduction targets, we've been having those discussions even in our team at Intel and the challenge is there really isn't a way to attribute any of those reductions to just us and our work. Which kinda breaks the OKR process, almost like greenwashing - claiming carbon reductions that are nothing to do with you, or at least not only you.

I do think however it would be good to calculate in general, as in calculate in total the carbon emissions due to software, who knows maybe after someone does that research it becomes possible to measure specific things that reduce that total. Maybe we need to fund/drive some research in this space?

[tmcclell]

Would that include carbon reductions due to aligning SCI as well? If I apply SCI to a scenario and use patterns to reduce the amount of carbon, wouldn't that be due to our work?

[jawache]

@tmcclell i think it's hard to really say it's because of only our work. For instance what if one of our patterns was a copy of an AWS pattern (some are), we just put the effort to get it into one place with consistent formatting. Do we get to make the claim of carbon reduction, or does AWS?

So for me it's how much can we defensibly prove was due to us alone. I agree would be good to prove some carbon number reduction but I'm not sure how to measure and report that in a way we can defend if challenged 🤷‍♂️

Part of me thinks things might get easier if we host our own APIs for things? So if we hosted an API people could use to measure carbon emissions of software, we could then have some actual data to prove something - "X tones of carbon calculated using the SCI".

If you don't mind I'd like to break this out into it's own discussion topic, it comes up a lot and I think would be good to have a consensus based view on the whole idea of claiming carbon reductions from software. Like how should we judge others and ourselves when they make claims of carbon reductions due to software optimizations?

[tmcclell]

Agree @jawache that we should continue to discuss and address - and good to break it into it's own thread. I've also gotten multiple pings around this too.

[jawache]

@Sealjay & @danuw - OBJ4 I think speaks the most to Open Source WG. Thinking through the one KR about measuring software emissions, every current project (ca sdk, carbon ci and the new carbon ql) is all about measuring software in one way or another so i think that one KR speaks to all of your work. It's fine to have one KR but if you feel another would help focus attention let me know.

[Sealjay]

That's correct! Here's my attempt to answer the Qs and write a good KR.

• What does success look like for this project?

Waiting on @KnowledgeRatio to create the PR-FAQ, but in my words, success would be the existence of a Carbon Data Interchange Spec that the GSF adopted and used on its own projects.

• If it existed would it make building green software easier, or would it make building green software more of a priority for organizations/teams?

If the GSF adopted and used this standard, it would make building green software easier, because GSF projects could use the existing data entities rather than spending time designing new entities.

It would make it easier for organisations outside the GSF to report and consume carbon data, again so they can reduce time in creating solutions to solve the climate problem, rather than fiddling with the underlying data formats.

• If lots of people were to use the project, what would this change?

If lots of people were to use this project, it would improve data quality, reduce time to create new solutions, ultimately saving money & resources, and speeding resolution of the climate emergency.

• Are there any non-binary ways of measuring the success of this project? (I know the SCI KR is binary, I've asked for help on that one! Really should be something you can track as a %age)

% of GSF projects producing and consuming carbon data that conform to the Carbon Data Interchange spec?

[jawache]

Great thanks @Sealjay I think that defo makes for a great OKR for the actual project itself!!!

My personal feels is that we can and should cherry pick a few KRs from some of the projects into the top level OKRs, but perhaps we should wait for a project to become graduated before picking one of it's KRs. So maybe wait till this project has some maturity then add it to the top levels. WDYT?

[Sealjay]

That makes sense, I agree.

[Sealjay]

Comment in OSS WG call from @dtoakley - could we have a metric for # of organisations that have adopted GSF tooling, to ensure that we don't hit the metric from one org adopting this on 100 apps.
Green-Software-Foundation/opensource-wg#66

[jawache]

Just circling back to this. So after working with PMs to actually try to capture and record these numbers we've had to adjust a few for practical capturing purposes and the tooling OKRs are particularly problematic. E.g. the metrics right now we don't really have any way of easily measuring. I was thinking perhaps if we hosted a service like carbon aware sdk or the carbon ql we'd be able to figure this out from our logs, but they doesn't exist so no point measuring something that doesn't exist. If we hosted a service we'd be able to calculate both number of apps and number of individual organizations.

This is a totally vanity metric and one we should work hard to move away from ASAP but what about something like "number of github stars for our open-source/tooling repos" - we could easily measure, it is somewhat something that points people in the right direction. It would drive activities that get eyeballs on our open source repos.

Thoughts?

[jawache]

@Henry-WattTime & @atg-abhishek - Standards WG. KR 4.2 isn't ideal, I'd prefer something where we can measure progress in a less binary way towards our goal (In ISO vs not in ISO is binary) open to suggestions or we can start with this and adjust later.

[Henry-WattTime]

The only thing we can really control is our response to ISO comments? Could we talk about the process instead of a deadline? Or instead progress and updates to the specification? We could also refocus on the guidance documentation we are creating and the case studies? Thoughts?

[jawache]

@Henry-WattTime I agree I never liked this OKR (that I proposed ;) ) I do however really like the idea of an OKR around published SCI case studies.

One way I really like thinking about OKRs is if this was the only way you had to influence the work of others, what OKR would you choose? As in you couldn't even speak to them and given them instructions, imagine setting an OKR was the only way you had to communicate with your team to direct their work.

If we set an OKR of "20 SCI Case Studies published by end of 2023". Imagine a dozen people are going to run off and try to maximize that number. It's a way for you to indicate to the army of volunteers what's important to you. You're not going to micromanage them, they own their own tasks, they come up with their own ideas and methods, you just try to tweak their work with the wording of an OKR.

Hope that makes sense?

[Henry-WattTime]

I like the idea of case studies because it requires people to implement the SCI (so we get the advantage of stress testing it and people using it) and it forces people to show their work making it more accessible for other people? Instead of apps just doing it, it's apps showing that they are doing it. Does that make sense? Even better would be case studies that have implemented one or more of the patterns so that they can show before and after. That's a bigger lift, but it rolls up a lot of goals (people using the SCI, people deploying the patterns, and people showing it is possible to implement the score).

[Henry-WattTime]

WG:
Proposed ideas:
-Number of orgs that have adopted SCI as a KPI (@GadhuNTTDATA )
-What is an app? Is is an or measuring or a case study? Can we make this clearer?
-So many enablers within GSF (i.e. patterns, sdk) (@srini1978 )
-Use cases and apps adopting might lead to emission reducing (@tmcclell )
-Release 2-5 New Open source projects (make sure projects under development are completed) (@navveenb )
-Asim has tracking spreadsheet for OKRs