Q & A
Our hosts Heather Fleming (Sr. Director, PMO, Gilt), Nate Van Dusen (Engineering Program Management Director, Trulia), and Maira Benjamin (Director of Engineering, Pandora) answered the top questions from this presentation. Read on to learn more about their agile success stories.
Q1: How do you measure "success" in a software development effort?
Heather: For each initiative, we work with each team to define their KPI (Key Performance Indicator). This KPI is the measurement of the success of that particular development effort. Where it is difficult to do this, we look for other tactical metrics to measure success based on the criteria of the project. An example at Gilt might be to increase the number of orders by X% between 12pm and 1pm, with X being the KPI's target.
Nate: I view success as the ability to constantly and quickly deliver new or incrementally improved features to customers.
Maira: Succes for us is measured in terms of our end user satisfaction. If the end user that we are working to create the product for is happy with the result, then we have success.
Q2: Just interested how people have found scaling Agile? How have the different companies done it, and has Agile itself helped scalability or not?
H: We work to maintain small, "pizza-sized" teams that are self-sufficient and can execute on their specific initiative -- which means as we scale, we end up having a lot of teams! We also have a centralized PMO which helps with cross-team dependencies and execution. Agile itself is definitely scaleable, it's the processes that needs to adapt.
N: Scaling is one of agile's greatest challenges. When a company experiences major growth, processes that were once successful can start to break down. Many times as a company grows, so do the size of the projects or teams. Organizing teams and work in a way to avoid cross functional needs becomes a challenge. Agile starts to face challenges the minute you start to have cross team dependencies.
Starting about a year ago there has been a growing interest in agile at scale or "enterprise agile." Tools are emerging such as Portfolio for JIRA to manage higher level agile needs and requirements while still feeding them into a normal agile processes. There are also methodologies emerging such as "SAFE" (Scaled Agile Framework) to try and tackle the issue from a process standpoint. I feel that it is still early days for agile at scale, but am very passionate about the topic and excited to see where it goes!
M: We've managed to have new teams adopt agile. Our teams are small. No bigger than 10 people on a team. That includes all of the various roles.
Q3: Have you had technological limitations or problems trying to apply related agile methodologies like continuous delivery or fast releasing overall?
H: At Gilt, we do continuous delivery, have a suite of microservices, and we dark release to production rather than going through staging for many types of projects. These have been in tandem with our agile methodology rather than limitations or problems. For those projects that require a staging environment, the difficulty has been keeping that staging environment up-to-date and working properly in order to test and release quickly. We are experimenting with a sandbox-type environment to solve this problem currently.
N: Trulia was only recently getting into the business of continuous delivery/deployment. Only one team was working in this manner and they developed custom flows using Jenkins, Bitbucket Server and JIRA Software with automatic transitions to enable this flow.
M: We practice continous integration and delivery. We haven't found any blocking factors for us in terms of agile. In fact, agile helped us create the continous integration and delivery mechanisms. We iterated on that, as well.
Q4: Do you establish a timeline? Like a traditional project plan in a tool like MS Project? If not, what schedule guidance do you use?
H: For an individual project, we create a "project brief" which is a tool to gain alignment as to what we're focusing on with our partners and stakeholders. In this brief, we "manage by milestone." This means that we will come up with what we believe the major milestones of the project to be, and for each we will give a very rough estimate in sprints. As we execute the project, we are constantly updating that milestone schedule and communicating the changes. As far as longer-term roadmapping, we focus on what is in progress right now, and what is the project coming up next – and stop there. We have found that if you try to predict too far out, you are wasting a lot of time as things will undoubtedly change before you get there.
N: Traditionally sprints are the only agile timebox, which are typically 2-3 weeks long. Trulia however has created quarterly retrospectives and planning sessions to create a quarterly view of epics. We use Portfolio for JIRA to manage the epic backlog at a quarterly level, which feeds into JIRA Software.
M: We haven't established a timeline for most of our projects. We are firm believers of high quality deliveries. That is usually in conflict with a "date."
Q5: Do you see agile working best for a SaaS solution software vs. a bespoke solution software?
H: It works for both. The agile principles are a set of behaviors -- it can work for any type of project (even for self-improvement!)
N: I see it working the same in both types of solutions.
M: I think it can work for both. But, I can't speak to it since we don't have SaaS solutions.
Q6: What's the conformation of your teams at each company? I mean devs only, devs + PO, devs + BA + PO, etc...
H: It is different for each team. We build teams based on the "ingredients" or qualities of each individual -- not on a set of titles. We make sure each team has the qualities to be successful, such as "coder" or "organizer" or even "cruise director" (the person who makes things fun!)
N: We have a product owner, scrum master and then team members. The members usually have a tech lead on the team.
M: We have a product manager, QA engineers, business analysts, scientists, and software engineers.
Q7: Is quality ensured by using iterative agile methodologies or rather because introduction of TDD / BDD?
H: At Gilt, ensuring quality is one of the "ingredients" that a team needs to have in order to be successful. How the team does this is up to them to decide. Ultimately they are responsible for ensuring quality, so if that means they are using TDD, or running monitors and alerts – they figure out what they need and make sure it gets done.
N: I think it is both. In agile the idea is to build quality in everywhere. If product owners are writing great user stories with acceptance criteria, then that enable developers to code to success and also allows QA to start building test cases from the start of or before development. It also enables a true partnership between QA and developers.
M: It is both for us. We really drive the point that product owners/managers need to have good stories for the best results.
Q8: What are the key steps creating an environment where sharing information is more valuable than owning information? Break down the "I know this and you don't therefore I'm in control"?
H: The first key step is to exhibit the behaviors that you want to see others adopt in the rest of your organization. So, take it upon yourself to start sharing information. Tranparency, visibility, and lots of communication can start to break down those walls. If you can't get traction, find someone who has influence in the organization and get them on your side. Propose an experiment to just "try out" for a short period of time, and set up your experiment to win. Share the wins broadly and invite others to try it out as well.
N: This is a difficult task anywhere but my belief is that communication problems can be tough to solve, so I start with creating visibility. As long as people are making decisions and taking action based upon the exact same information, then you have a higher chance of success even where communication issues exist. As far as individuals wanting control, I also try to encourage a team first culture instead of an individual focused one. There are many ways to go about this, but agile fundamentals are a good place to start. If everyone is empowered and able to express their thoughts, choose their work, estimate their work, break it down into sub-tasks etc, then this helps.
M: We don't have egos at Pandora. It's one of our culture's strengths. Everyone is collaborative and encouraged to be so. We also hire that type of profile for the teams across the board. I think if you start with that, communication is a lot easier acorss the board.
Q9: Maira, you mentioned that the teams coming from academia had more trouble "being agile with agile." As someone who is transitioning from academia to business development, I am curious about common obstacles that academics encounter in agile.
M: The obstacles for people coming from acadmia are the following: (1) unfamiliarity with the concept, (2) inability to break down the project into smaller chunks, (3) tendency for academics to never feel that the work is a "product", (4) Inability to estimate the tasks once broken down, (5) not understanding what a minimum viable product looks like, (6) scaling their work into production. Now, having said all that, once the academic becomes more familiar with agile, they are more likely to embrace it for future projects. You have to remember that academics take many years to produce a thesis or a concept, so it's very jarring for them to step away from that paradigm and produce things in a shorter block of time.
Q10: What do you think about following agile principles not in a team but as a standalone developer? Can I expect benefits from it or is it more like to break a butterfly on a wheel?
H: Yes totally! You can expect a lot of benefit from following the twelve agile principles as an individual. They're guidelines on how to behave – and you can figure out how you want to layer in an execution process on top of that. It doesn't have to be Scrum, necessarily. Find what works for you!
N: I think that there are many strong takeaways from Agile even for an individual developer. Just the idea of creating an MVP (minimum viable product) version of software and iterating quickly on it is a huge win. In addition I think that building quality in... doing the right thing at the right time can help ensure quality code and minimize technical debt. It's also a leg up should things be successful and a team develops around the individual!
M: Oh! Well I use agile in my personal life. It's helped me with my creative pursuits. The tenets of agile can help everyone with their personal projects. I've been able to plan out many trips, write more, and tackle some of the big projects that wouldn't get done unless I applied the principles.