Sub-Award Assessment

by Lisa Lampman
March 22, 2004
Bill Freeman, NRC Moderator
Actual Transcript – 60 minutes

Moderator (Mar 22, 2004 2:55:26 PM) Welcome everyone -- we are waiting for a few minutes for everyone to get online.

Moderator (Mar 22, 2004 3:02:30 PM) Welcome again.

My name is Bill Freeman and I am the NRC Knowledge Manager, and today's Moderator. We are pleased and excited that you have agreed to participate in our third "Ask the Expert" online training on Sub-Award Assessment Planning.

First off, don't worry - this is an easy thing to do. There are three roles in this session: Lisa Lampman is today's Expert, I am the Moderator, and you are the discussion participants.

Lisa will take the lead and present the session. This includes providing information, answering questions and feeding discussions. I am the Moderator; I am like a switchboard operator. When you ask your questions they are presented to me, (you won't see them). I will then review your questions and send them on to Lisa. For most questions, I will post them live for all of us to see and for Lisa to respond.

You are the discussion participants. This session is for you. When you want to ask questions, just type them in the bottom panel on the screen. Now don't bother taking notes. This session has afterlife! Following this discussion, we will email you a full transcript of the session (simple edits for typos and grammar). If you experience simple connection difficulties, you can call Yvette Green here at National Resource Center at 703-752-4331.

Lisa Lampman (Mar 22, 2004 3:04:45 PM) Thanks so much, Bill, it's a pleasure to be with all of you this afternoon to discuss Sub-Award Assessment Plans.

Lisa Lampman (Mar 22, 2004 3:05:34 PM) Let's start by reviewing some key information about due dates:

Only 2003 grantees need to turn in a sub-award assessment plan;
If 2003 grantees have made some or all of their sub-awards, their plan should be turned in to the Federal Project Office by March 31, 2004;
If 2003 grantees have not yet made sub-awards, their plan should be turned in to the Federal Project Office within 14 days of making your sub-awards. You should request an extension from the Federal Project Office by phone or email prior to the March 31 deadline.
One original and one copy of your plan should be sent to Kelly Cowles, CCF.
Project Manager. 2002 grantees should include sub-award assessment reports in their semi-annual reports, which are due by April 30, 2004.

Moderator (Mar 22, 2004 3:06:20 PM) So only first year intermediaries need provide this?

Lisa Lampman (Mar 22, 2004 3:07:10 PM) Yes, 2002 intermediaries will provide sub-award assessment reports in their semi-annual reports.

Lisa Lampman (Mar 22, 2004 3:08:49 PM) The purpose of this session today is to assist CCF grantees in developing your sub-award assessment plans. Feel free to ask questions at any time.

Lisa Lampman (Mar 22, 2004 3:09:17 PM) There are three parts to your sub-award assessment plan. As needed, we can discuss each of these in more detail as the session progresses.

Organizational Profile Data: data collected on or from sub-awardees. Probably the easiest way to collect the data is from the RFPs that the sub-awardees submitted. If not all the information is available there, you may want to use an email or phone survey to get the information quickly. To save you time next year, be sure that all the profile data information is available from the RFP. Also, you may want to collect additional data for your own organization-such as the age of the organization (is it just starting out or has it been around a while?); its current 501c3 status; and if the organization has previously received public funds (local, state, federal).
Organizational Effectiveness: focuses on outcomes and indicators for each outcome. As an intermediary, your outcomes should focus on increasing the capacity of the FBOs and CBOs you serve. For the sub-award assessment plan, you will be developing outcomes and indicators for the FBOs and CBOs that receive sub-awards from you.
Qualitative Analysis: focuses on the stories of how individual sub-awardees used funds and Training and Technical Assistance (TTA) to significantly influence the lives of the people and the community they serve and/or improve their operations and organizational structure for increased effectiveness.

Lisa Lampman (Mar 22, 2004 3:12:12 PM) Also, as we get started, it might be good to review the definition of an outcome and an indicator. For the purpose of the CCF sub-award assessment plan, we'll use the following definitions:

Outcome: Positive changes in the FBO/CBO as a result of receiving the sub-award and TTA support.
Indicator: The specific, measurable information collected to track whether an outcome has actually occurred.

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:13:24 PM) Lisa, We have not yet made sub-awards. On 3-31-04, we have an assessment plan due. What do you suggest we do for our preliminary plan.

Lisa Lampman (Mar 22, 2004 3:14:39 PM) Good question, Ken. You don't have to turn in a plan by 3/31/04 if you haven't made sub-awards yet, but your plan should be turned in to the Federal Project office within 14 days of making your sub-awards.

Lisa Lampman (Mar 22, 2004 3:15:42 PM) You should request an extension from the Federal Project Office by phone or email prior to the 3-31-04 deadline.

Lisa Lampman (Mar 22, 2004 3:16:26 PM) So, you don't have to submit a plan by the deadline but you need to get approval for an extension.

Lisa Lampman (Mar 22, 2004 3:16:53 PM) Ken, does that answer your question?

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:17:32 PM) Yes. Thank you and I will try to get an extension!

Moderator (Mar 22, 2004 3:19:27 PM) Lisa, you introduced three areas to focus on: organizational profile data, organizational effectiveness, and qualitative analysis. Can you tell us more about organizational effectiveness? How do you measure things that seem to be by their nature immeasurable?

Lisa Lampman (Mar 22, 2004 3:20:58 PM) Gladly, I think that's the toughest of the three parts of the sub-award assessment plan to work on. The organizational effectiveness part focuses on outcomes and indicators for each of the outcomes. Your sub-award assessment planning should focus on identifying outcomes related to increasing faith-based and community organizations' knowledge and skills and bringing about positive change in the FBO/CBO.

Lisa Lampman (Mar 22, 2004 3:22:04 PM) Again, as an intermediary your outcomes focus on increasing the capacity of the FBOs and CBOs you serve. For the sub-award assessment plan, you will be developing outcomes and indicators for the FBOs and CBOs that receive sub-awards from you.

Moderator (Mar 22, 2004 3:23:34 PM) Does that mean that you shouldn't focus on the programmatic outcomes (i.e. less homelessness and hunger)?

Lisa Lampman (Mar 22, 2004 3:24:53 PM) Exactly. To define outcomes for the sub-award assessment plan, it is best to focus on the impact that your training and technical assistance and sub-awards have made on the FBOs and CBOs you serve.

Lisa Lampman (Mar 22, 2004 3:25:39 PM) Let me give you an example of some outcomes that might be appropriate for you to set for your sub-awardees. The FBO/CBO has:

Expanded and enhanced service delivery;
Increased fund development capabilities and opportunities;
Increased financial management expertise;
Strengthened organizational structure;
Improved coordination and/or collaboration with others;
Enhanced leadership and management capabilities.

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:25:49 PM) Is there a definitive means of measuring "increased knowledge and skills" besides possibly a pre- and post-test.

Lisa Lampman (Mar 22, 2004 3:27:50 PM) Great question--you are talking about indicators and using "increased knowledge and skills" as your defined outcomes. Here is an example for considering the success in fund development programs:

Did they expand and diversify funding sources?
Did they increase the number of grants they submitted?
Was there an increase in the percent of in-kind goods and services donated from baseline?
Was a long-term strategic plan approved and implemented?

Lisa Lampman (Mar 22, 2004 3:31:46 PM) With an outcome such as how a FBO/CBO has increased its ability to define and measure outcomes for a sub-awardee, you might consider these indicators:

staff complete training and implement outcome measurement plan for agency;
outcomes and indicators defined for the agency;
program improvements made on outcome findings.

Lisa Lampman (Mar 22, 2004 3:32:03 PM) Ken, is that helpful--what you are looking for?

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:33:32 PM) Lisa, Yes! Can you also give an example of an indicator for enhanced leadership and management capabilities?

Lisa Lampman (Mar 22, 2004 3:34:50 PM) Here's some thoughts:

development and implementation of a strategic plan;
board defined roles and functions;
board that meets regularly and reviews budget and funding plans, etc.

Lisa Lampman (Mar 22, 2004 3:36:58 PM) Because we're talking about indicators, let's take a closer look at the definition of indicators--the specific, measurable information collected to track whether an outcome has actually occurred.

Lisa Lampman (Mar 22, 2004 3:37:36 PM) For your sub-award plan, you really only need 1-2 indicators per outcome. And you only need 1-2 outcomes per sub-awardee. I think it's best to keep it simple!

Lisa Lampman (Mar 22, 2004 3:38:42 PM) It's good to remember that the purpose of an indicator is to help you know whether the outcome(s) have been achieved. Specifying one or more indicators for outcomes requires deciding:

the specific, observable, measurable characteristic or change that will represent achievement of the outcome;
the specific statistic(s) (e.g., number and percent attaining outcome) the program will calculate to summarize its level of achievement

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:39:21 PM) Great! I am on the right track for the outcome of enhanced leadership and management capabilities. We had planned to measure things like:

Does the organization have a strategic plan;
Are the funding strategies in line with the strategic objectives (mission) for the organization;
Are the board minutes up to date, etc.

Lisa Lampman (Mar 22, 2004 3:39:27 PM) Some outcomes are fairly easy to observe and measure – such as increasing funding opportunities. You can measure that by determining how many grant or funding requests were made by the organization and how much has been raised. However, some outcomes are harder to measure – such as increasing leadership or management capabilities.

Lisa Lampman (Mar 22, 2004 3:40:32 PM) Here are some helpful questions to ask when you set out to identify indicators for your outcomes:

How can I see the change? (direct observations)
How can I hear the change? (interviews, focus groups)
How can I read the change? (surveys, records)

Lisa Lampman (Mar 22, 2004 3:42:01 PM) These questions can assist you in determining if indicators are measurable and if that information is accessible to your team. If the information is difficult to measure or not easily accessible, you might want to think about other indicators.

Moderator (Mar 22, 2004 3:43:09 PM) Lisa anything else?

Lisa Lampman (Mar 22, 2004 3:43:21 PM) Here are some tips for determining indicators:

Focus on indicators from which you can obtain data;
Apply the "means that" rule -- the accomplishment of an indicator "means that" the FBO/CBO has achieved the outcome;
Identify indicators that provide the most useful information about the outcome;
Choose indicators that are most feasible to measure with the time and resources available to your organization;
Select 1-3 indicators per outcome.

Lisa Lampman (Mar 22, 2004 3:43:52 PM) Would folks like me to give them some examples of indicators for different outcomes? Would that be helpful to you?

Ken: Louisiana Association of Nonprofits (Mar 22, 2004 3:44:52 PM) Yes.

Lisa Lampman (Mar 22, 2004 3:45:59 PM) Let's try the outcome of expanded and enhanced FBO/CBO service delivery. Here are some possible indicators:

15% increase in new clients served (from baseline);
Expanded geographic area of service;
New and diversified services provided to clients.

Lisa Lampman (Mar 22, 2004 3:46:31 PM) Another one is strengthened organizational structure:

501c3 application completed, submitted, approved;
Annual strategic plan developed and implemented;
Board members attend board training sessions and develop job/role descriptions.

Moderator (Mar 22, 2004 3:47:30 PM) Since we only have a few minutes left, tell us about the qualitative analysis part.

Lisa Lampman (Mar 22, 2004 3:48:13 PM) Glad to do so. The qualitative analysis is a detailed narrative used to demonstrate any of the progress cited in the outcome and indicator part of the plan. It's an opportunity for intermediaries to tell their stories -- how sub-awardees used funds and TTA to significantly impact the lives of the people and community they serve and/or improve operations and organizational structure for increased effectiveness.

Lisa Lampman (Mar 22, 2004 3:49:22 PM) These stories make CCF's work tangible by describing how an individual FBO/CBO benefited from your intermediary's TTA and sub-award.

Lisa Lampman (Mar 22, 2004 3:49:46 PM) Be sure that you keep the focus of the story on the impact of your project on one or more participating FBOs/CBOs -- how they changed as a result of involvement in the CCF program and what impact that made on the FBO/CBO and the lives of the people/community it serves.

Lisa Lampman (Mar 22, 2004 3:51:03 PM) If you are a 2003 intermediary, you may not have a lot of stories to tell right at first since you are just getting started.

Lisa Lampman (Mar 22, 2004 3:51:57 PM) But as you provide the sub-awards and training and technical assistance, be sure to be looking for the stories that capture the impact that your TTA and sub-award help has on the FBO/CBOs you serve.

Lisa Lampman (Mar 22, 2004 3:52:33 PM) Don't worry. The stories don't have to be long--but they should show impact and be memorable. I might suggest that you think about developing stories in three (3) main parts:

Lisa Lampman (Mar 22, 2004 3:53:48 PM)

Story lead: presents the situation, need and the opportunity of the FBO/CBO
Main body: describes how sub-awards and TTA helped the FBO/CBO
Summary: describes the end result or benefit for the FBO/CBO as well as for clients and community. You may want to use quotes or examples to help summarize and capture the story.

Lisa Lampman (Mar 22, 2004 3:55:23 PM) A good example can be found in the Feb 24 issue of the E-newsletter--the CCF dollars at work story on JVA's sub-awardee Covenant Cupboard in Denver, CO. Check it out.

Moderator (Mar 22, 2004 3:55:34 PM) Our time is getting short, let's have just one more question. Any burning questions?

Oscar: Louisiana Association of Nonprofits (Mar 22, 2004 3:56:10 PM) Let's say a participating FBO/CBO helped an individual who is homeless find a job; is that the kind of impact success story you are talking about?

Lisa Lampman (Mar 22, 2004 3:56:58 PM) Thanks, Oscar, good question. I would try to connect the story back to how your organization's TTA and/or sub-award made an impact on the FBO/CBO and how it in turn helped a client or the community.

Lisa Lampman (Mar 22, 2004 3:57:31 PM) Just a few summary thoughts for all of you:

Keep your plan focused on the FBOs/CBOs you serve through sub-awards;
Keep outcomes simple and realistic;
Keep indicators specific and measurable;
Make stories memorable.

Moderator (Mar 22, 2004 3:57:40 PM) Thank you for participating in today's session. As always, we are here to help you in anyway that we can. We are as close as a phone call or a Web click! Good-bye.

Lisa Lampman (Mar 22, 2004 3:58:18 PM) Thanks so much, Bill and all of you who participated. If you have any further questions, please feel free to call or email me--703-752-4331 or lbl@daremightythings.com.

If you have trouble reading this e-newsletter due to formatting issues, or visible HTML code, or if you would like to discuss content-related issues, please contact Bill Freeman, NRC e-Newsletter Editor at wjf@daremightythings.com.

Hypertext links and other references to non-CCF products and services are provided for information only and do not constitute endorsement or warranty, express or implied, by the CCF, DHHS, or U.S. Government, as to their suitability, content, usefulness, functioning, completeness, or accuracy.

Best of Index:

Search All Articles

Sub-Award Assessment