The Exchange, December 2004

Issue 11(4), December 2004

In this issue

Creating a role for yourself in developing research proposals
Part 2. Examining the RFP and the proposal

By Laurel K. Grove (l.grove@ieee.org)

Things to look for in the RFP

What specifically should you look for in the RFP? It's common among people preparing proposals to complain about RFPs, which at worst are incoherent and at best seem to have been written by committee. In fact, RFPs really are written by a committee in most cases, because many factions within the sponsoring agency have concerns that must be addressed. Sometimes these concerns, or their solutions, are contradictory. There may not be much that you can do about that, although you can point out the problem in the sponsor’s pre-proposal meetings or question sessions. However, you can take comfort in the fact that all of the other potential responders are handicapped by the same incoherent document. (There are projects that it’s best for someone else to win; the problems inherent in those projects may remove the winning organization from competition for more tractable projects.)

The problem of incoherence aside, what’s good about the RFP is the amount that it tells you not just about the problem to be solved through research, but also about the sponsor’s concerns and the way the contract will be awarded.

Deadlines

One of the requirements specified in the RFP is the proposal deadline. If you're used to projects with ever-extended deadlines, you're in for a shock: This deadline is real, and a proposal that comes in late is unlikely to be reviewed. Every responding group is subject to the same deadline. If a deadline is clearly impossible, for instance if the sponsor is unable to provide necessary background material in advance, you can bring up that problem with the sponsor immediately, and they may extend the deadline—for everyone who has received the RFP. In the usual competitive contracting situation, such as those involving the federal government, no individual or organization will receive any extension or additional information unless the same extension or information is given to all proposers. An extension because you didn’t get around to reviewing the RFP until two days before the proposal is due will not be granted. That is not unreasonable, since the sponsor has usually taken pains to allow adequate lead time for proposers to pull together their plans, and it has its own schedule to meet. If your organization wasted that lead time, that isn’t the sponsor’s problem. In addition, they have no reason to trust that an organization wouldn’t waste the project’s time and money if it can’t even respect the proposal deadlines.

In addition to that major deadline, there may also be subordinate deadlines, such as those for delivering a letter of interest, submitting pre-proposal questions about the project, and attending meetings to discuss the project. Missing these deadlines can be as fatal to a project as missing the deadline for delivery of the proposal itself. Note every date specified in the RFP and whether the actions specified for that date are mandatory or optional. Then plan accordingly.

Format

The RFP often details the format of the response. Follow that format exactly. There are several reasons for this recommendation: First, it shows the sponsor that you are willing to listen to them and to attempt to meet their requirements. Second, the reviewers will look for material only where they expect it. If you place that material somewhere else instead, you probably won’t get credit for having included it. Reviewers are inevitably in a time bind and will not search for information that they can't find in its expected location. Sometimes adherence to format requirements is crucial to winning funding; it may be the criterion for making the first cut in the review process. As the review cycle begins, each incoming proposal is checked by a clerk whose job it is to reduce the review load by flatly rejecting any proposals that fail to meet the format requirements. Even the best science will never occur if the proposal is rejected during this first, purely administrative cut.

Language

Something that you will notice in the RFP that might escape the researcher is the sponsor’s preference for language. English offers innumerable synonyms, but a group of people working together usually share a single set of terms and may not even recognize others. In reviewing the RFP, watch for the sponsor’s preferences, as expressed in the terminology used in the RFP, so you can use compatible wording in your proposal. If you know that the researchers’ vocabulary differs from the sponsor’s, point out the alternative terms early so the researchers can get comfortable using the sponsor’s words as they write the proposal and so they are prepared to use these words in any pre-decision presentations and proposal defenses.

Selection criteria

Most RFPs identify the sponsor’s selection criteria, usually at a fairly high, non-specific level; they often indicate the general rankings of importance of various issues. Make sure that researchers understand the relative importance of these issues and allocate their time accordingly. By reading the RFP carefully, you can also pick up on more specific concerns. These may not be the biggest point-getters, but they may serve as tie-breakers between two proposals with similar scores. Remind researchers of these issues, and help them address each one. Although this detail may not be specifically mentioned in the selection criteria, it would not be surprising to find that if two researchers propose essentially the same research plan and have basically the same credentials, reviewers are likely to give more points to the one that is written using the language of the sponsor and that includes examples that link directly to the sponsor’s past concerns or problems. Look for such links as you gather the material required to write the proposal.

Costing

Virtually all RFPs require the inclusion of a detailed budget, but the format required may vary widely. For example, some sponsoring agencies refuse to pay "overhead" costs or set a low cap on the overhead rate. In these cases, the services usually categorized as overhead (such as clerical services and information systems) must instead be listed as line items to ensure that they will be paid for. Alternatively, they might be listed as a "matching contribution" from the researcher's employer. There may be multiple categories of overhead to cover such items as fringe benefits on labor (these may be merged into the labor cost as a “burdened” rate) and administrative assistance, library services, or shared equipment. A sponsor may limit overhead because they have seen too much administrative waste or because they do not recognize the need for shared services; the researcher may not realize that even shared services must in fact be paid for by projects. Nevertheless, all services used by the project must be accounted for somehow. Each item in the budget should be defined and justified; failure to follow guidelines specified in the RFP may lead to the rejection of the proposal.

Basis for decisions

The biggest reason for reading the RFP is to discover what is most important to the sponsor. For instance, the RFP may spend as much ink on management questions as on the science, but the researcher will consider management uninteresting and a non-issue; "management is management, period". Unfortunately, many a sponsor has been stung by a poorly managed project that spent all its funds and produced no results, that could not account for how the money was spent, that changed principal investigators as if they were in a revolving door, or that used inadequately trained staff who misused resources and put the sponsor at risk of lawsuits. Once bitten, twice shy; the sponsor will add management requirements, sometimes to the point where the quality of management controls becomes the deciding factor as to who wins the grant. By reading the RFP, you can rely on your detachment as a technical communicator to prevent your being led off on a trail of imagination by the excitement of the science, and you can determine where the focus of the proposal must be to win the funding.

Things to look for in the proposal

Budget

Make sure that any institutional overhead is included in the form required by the RFP, and that these items do not exceed the specified caps. Ask yourself: Does the budget cover the real cost of the project, or does it merely match the sponsor’s price cap? Sponsors are justifiably skeptical about proposals that request exactly the available funds. Also keep a watch to ensure that your organization’s real costs are fully covered. I am currently working with a group that is moving out of a university setting. Unfortunately, the principals do not realize how protected they have been at the university, where they freely borrowed space and materials from colleagues and apparently reconciled or recovered underestimated costs with money from some invisible internal source. They accepted low pay and long hours there, completing projects on their own time and using their own limited resources without complaint. But now they are having difficulty with labor rules that limit the number of hours that staff can legally work and with sponsors who want to know exactly where the money is going.

One issue that can be problematic involves equipment purchases. Some sponsors refuse to pay for such purchases; others insist that the equipment be turned over to them at the end of the project. Make sure that it is clear who will hold title to any equipment purchased, and that this decision is in accord with the sponsor’s requirements.

Other concerns include making sure that expenses can be justified. This requires that you cross-check between the budget and the research plan to ensure that there are no contradictions or gaps. If costs are based on data rather than arbitrary calculations, the proposal becomes more credible.

Research plans

Are the activities that define the research plan clearly specified? Read the proposal as a reviewer might: critically and skeptically. Can you tell exactly what the researcher plans to do? Can you tell what the money will be spent on, and is it something that the sponsor is interested in supporting? The sponsor expects to know exactly what the researcher plans to do, even if the results of those activities are necessarily unknown. The researchers should give a clear idea of how they will approach the investigation, then indicate how many times they will run a given test and calculate what it costs to set up and make each such run. “Going into the unknown” is nothing more than an excuse to avoid planning, and it is not a valid one. Sponsors are no more willing to commit to paying “whatever it takes” than we would be for home remodeling or car repair.

Also make sure the plan is realistic. Check the schedule to be sure that staff are not assigned to work 80-hour weeks or holidays. Like the budget, the schedule must be realistic, ideally based on data collected during similar prior work.

Format

Does the proposal exactly match the RFP's requirements in terms of organization, length, font size, and anything else identified in the RFP? RFPs tend to be quite detailed, and as noted earlier, failing to meet any of the requirements can cause the proposal to be rejected even before a review committee considers its scientific merits.

Language

Is the language of the proposal the language used in the RFP and hence the language familiar to the reviewers? It isn’t enough for you to create a glossary or concordance and let the reviewers match the terms. It also isn’t enough for you to use the concordance and simply translate. The researchers themselves must be willing to acknowledge and use the sponsor’s terms; anything else suggests the researchers’ disrespect, even disdain, for the sponsor.

Appropriateness

Is the proposed research plan in line with the issues identified by the sponsor? After spending time pulling together a research plan and determining what would be their best choices if money were no object, researchers sometimes propose those choices as the minimum that would be required to solve a sponsor’s problem. The sponsor is generally more interested in solving the problem expeditiously than in having a possibly better solution that entails higher costs, longer development time, or other risks.

Tone

Is the tone appropriate? I have read proposals in which disgruntled researchers felt the urge to admonish their sponsors for having failed to provide full funding for a project in the past. That may have made the researcher feel better, but it seemed ungrateful and did nothing (at best) to improve chances of winning further funding. No one ever has enough funds; whining about the lack is a waste of time. Selection criteria often include at least one subjective category to allow the reviewers to factor in their feeling about the researchers; a carping or hostile attitude could lose points for the proposal.

Completeness

Is everything that is included in the proposal relevant, and is everything that is relevant included? Is every requirement indicated in the RFP addressed by an appropriate response? Overworked reviewers may look for any discrepancy that can label a proposal as non-responsive to their needs and let them eliminate it from consideration. Make sure that your proposals make the cut.

Conclusions

As technical communicators, we have much to offer our researcher colleagues as they seek research funding. We can improve their chances of winning funds by making sure that their proposals focus on and meet sponsor needs. We can sometimes even improve the quality of their research by making sure that they haven’t set themselves up to run short of time or funds by neglecting to plan fully. The proposal stage is the best time to correct weak research plans, before impossible commitments are made. Our experience with other projects means that we can help researchers negotiate not just the sponsoring agency's requirements but also any internal institutional hurdles; our past work with administrators gives us knowledge of the process and sometimes wins us favors that we can call on to help smooth the way for future proposals.

Laurel K. Grove is a Senior Member of STC's Willamette Valley Chapter (Portland, Oregon). She first worked with research proposals as an archeologist, both writing proposals to seek research funding and writing RFPs and reviewing proposals to fund archeological projects. A career change later, as an editor in the environmental sciences (at Battelle Northwest, Battelle Pantex, and Oregon State University’s College of Forestry), Laurel realized that the quality of research depends on the quality of the proposal and began to work on every proposal she could. She also served on STC’s Research Grants Committee (1997–2002; chair 2001–2002), reviewing numerous proposals in our field. After a stint with a high-tech company for which she was developing an average of two new proposals every week, Laurel is taking a hiatus and working with slower-paced non-profit groups near her home in Salem, Oregon.

Editorial: Check your assumptions at the door

by Geoff Hart (ghart@videotron.ca)

"If you torture data sufficiently, it will confess to almost anything."—Fred Menger, chemistry professor (1937–)

Since not long after the beginning of the scientific revolution, there's been a growing societal consensus that numbers are important, and that indeed, anything important can be grasped if only a number can be assigned to it. (This is a more specific instance of the general rule that humans love categorizing things, and that anything we can't fit into a category makes us feel uncomfortable.)

To a scientist, everything is a metric: something that can be measured. To me, one of the really interesting things about metrics is not just what they say about the thing being measured, but what they say about the measurer. Consider the example of intelligence quotient (IQ) testing. Early in the 20th century, psychologist Lewis Terman popularized the IQ test originally developed in France by Alfred Binet. As recounted by William Poundstone (2003, p. 27) in a refreshing account of the merits and demerits of intelligence testing, Terman found a few inconvenient results during his initial broad application of what eventually became known as the Stanford–Binet IQ test. On average:

As any good scientist would do, Terman asked himself what these differences meant. But unfortunately, Terman made a key assumption that led him astray in his attempts to answer this question: he assumed that the test was biased because it didn't fit with his expectations. The test was indeed biased, but not even remotely in the way he expected, and Terman went completely off the rails in making two additional assumptions:

In short, Terman assumed that men and women were of equal intelligence, but that "Whites" were more intelligent than other races. A test that could have provided some interesting and important insights into the nature of testing and intelligence (why do the observed differences exist? what do they tell us?) became instead an effort to support pre-existing prejudices. Since then, IQ testing has often become a rote application of standardized tests that increasingly demonstrate nothing of any real interest other than the ability to take a particular IQ test. The developers of the Scholastic Aptitude Test (SAT), to name one infamous example, have spent enormous effort trying to produce a truly representative test, yet it's still true that you can dramatically improve your SAT score simply by studying how to take the test. How objective can any such test be if a significant proportion of the score depends on whether you've been trained to take the test?

With the benefits of a century of hindsight, it's natural for us to assume that any competent scientist would have thought carefully about the early IQ findings, and would have carefully designed hypotheses to let them test various interpretations of the findings. Eventually, such an inquiry would have led the investigators to more objective and interesting conclusions—such as insights into disparities in how men and women are taught science, and in how and whether different ethnic populations are being failed by their educational systems. (Of course, that assumes you accept a definition of "failure" based on test scores, which is something of a questionable assumption.)

Currently, there's a growing consensus that intelligence is a multidimensional concept with many facets; for example, psychologists now distinguish between emotional and mathematical intelligence, among other dimensions. Now that's interesting! Complexity is often far more fun to explore than simplicity when it comes to understanding how the world works.

By itself, however, the notion of multidimensional intelligence still fails us through the shackles it places on our thinking. If we assume that the test for (say) emotional intelligence is valid, we may fail to ask the important next question: What is the source of any observed differences in test scores? Then there's the follow-up question: What implications does this have? For example, we might want to spend some time in our high schools teaching teenagers to understand and deal with their emotions. If the goal of our educational system is to prepare our children for adult life, we certainly seem to be neglecting an entire dimension of their intellectual development: the emotional side.

It can be remarkably difficult to identify one's assumptions; this is why, among other reasons, I can make a very good living as a technical and substantive editor of scientific manuscripts produced by many people with IQs arguably much higher than my own. It's also why science journals conduct time-consuming and expensive peer reviews. But difficult is not the same as impossible, and the difficulty becomes much less if you're willing to ask yourself one fundamental question: "What conditions are required for this hypothesis to be valid?" Better still, we can ask a friend or colleague to ask that question for us, since their assumptions are likely to be sufficiently different from our own that they're not blinded to those assumptions.

I suspect the same question has important consequences for us, as communicators. What assumptions do we make about our audiences when we choose what to communicate with them, and how to perform that communication? How could we test those assumptions? If those assumptions are wrong, we fail to communicate successfully. If the results of the tests don't conform with our prejudices, we may have discovered something significant about more than our own prejudices. Whatever form of communication you perform, I encourage you to ask yourselves these questions—to check your assumptions at the door and see what interesting new things you might discover thereby.

Coming attractions

If all goes as planned, the March 2005 issue of Intercom will be a special issue on scientific communication. Watch your mailbox! I hope the articles in that issue, including (as you might expect) my own contributions, will inspire you to consider writing your own article on our profession, whether for Intercom or for the Exchange. Got an article idea? Drop me a line and let's discuss it!

E-mail or newsletter problems?

If you haven't been receiving e-mail from the Scientific Communication SIG's e-mail discussion group, please follow Douglas Adams' advice: "don't panic!" The volume of mail in this forum is very low indeed, and we often go months between messages. But when an occasional message does go out, I get dozens of bounce messages from people whose e-mail addresses have expired. When I can't contact the person at that address, I delete them from the list of subscribers. If you haven't received any mail from this group, but want to, please see the instructions on the last page of the newsletter for information on how to resubscribe.

If you or any of your colleagues haven't been receiving notification that the most recent issue of this newsletter has been published, please confirm with STC that your contact information is correct. Many people forget to inform STC when they change jobs or Internet service providers, and that means they won't receive any messages from our SIG. Because STC values our privacy, they don't give me access to the membership database, even though I'm the SIG manager. As a result, I can't help you change your contact information; you'll have to do this yourself.

Bibliography

Poundstone, W. 2003. How would you move Mount Fuji? Microsoft's cult of the puzzle. How the world's smartest companies select the most creative thinkers. Little, Brown, and Company, New York, N.Y. 276 p.

Transformation: help keep our SIG alive!

by Geoff Hart, Manager, Scientific Communication SIG (ghart@videotron.ca)

You've undoubtedly heard much sound and fury about the "transformation" exercise that STC is currently going through. If not, have a look at the transformation Web page (http://stc.org/transformation/) to get an idea of what STC is thinking and what has been accomplished thus far. I'm not a fan of such initiatives, since my experience with them has been that they often turn out to be equal parts Dilbert comic strip and meaningful effort to improve things. I won't bore you with my opinions on STC's initiative, but given the magnitude of the changes that will occur, I encourage you to research the issue and come to your own opinion.

There is, however, one area in which transformation will have clear effects: all STC "communities", whether structured as geographic chapters or the "virtual communities" of the special interest groups (SIGs), will be asked to "recharter" in 2005. Rechartering is something of a vague notion, but in essence, it's an effort to review why a community exists, who its members are, and how best to serve the needs of those members. As in this case, it can also become an opportunity to justify the existence of the community and continue its existence.

Membership in the Scientific Communication SIG has fluctuated between 600 and 1000 members over the years; at last count, there were 733 of us. Membership in our e-mail discussion group also fluctuates, but typically amounts to 80 members, few of whom ever post a message or contribute to a discussion. Currently, the only tangible evidence that our community is alive is this newsletter, and it's an increasingly strenuous effort for me to find new articles. In fact, past issues during my tenure as editor (http://www.stcsig.org/sc/newsletter/newsletter.htm) suggest that I'm responsible for roughly one third of the articles, and that another third come from people outside our SIG. That's hardly evidence of a vibrant community.

So... back to rechartering. Some time soon, STC will ask me to conduct a rechartering exercise to justify our existence. If we hope to continue to exist in 2005, citing membership numbers alone won't suffice to demonstrate that we're meeting a need. STC will demand tangible evidence that members are happy with our activities and that we should continue to exist. I'll need your help to convince them. Please send me your opinions on the following issues:

I look forward to your comments.

Parting thoughts

"Think of science as a powerful searchlight continuously widening its beam and bringing more of the universe into the light. But as the beam of light expands, so does the circumference of darkness."—Morris Meister, quoted by E.L. Doctorow

—

“The mind, once expanded to the dimensions of larger ideas, never returns to its original size.”—Oliver Wendell Holmes

—

"A good scientist is one who not only undersands what he's looking at and asks important questions but also one who wants to keep evolving his questions. Our job is not to get locked in."—Barry Chernoff, Professor of Environmental Studies, Wesleyan University

—

"Great ideas, whether true or false, do drive our research forward by focusing our thoughts and suggesting new pathways of exploration... Great theories are expansive; failures mire us in dogmatism and tunnel vision. I do not know the actual context of Goethe's famous dying words. Perhaps he was only asking for another candle, all the better to view the faces of his beloved one last time. But perhaps he was begging the Almighty for the greatest gift that fruitful theories can provide—Mehr Licht (more light!)."—Stephen Jay Gould, More light on leaves

—

“What does it mean to have an open mind? It is to find the essential balance between orthodoxy and heresy, between a total commitment to the status quo and the blind pursuit of new ideas.”—Michael Shermer, Skepticism as a virtue

—

“One's first step in wisdom is to question everything—and one's last is to come to terms with everything.”—Georg Christoph Lichtenberg

—

“It requires a very unusual mind to undertake the analysis of the obvious.”—Alfred North Whitehead

—

"Besides learning to see, there is another art to be learned—not to see what is not."—Maria Mitchell, astronomer

—

“We especially need imagination in science. Question everything.”—Maria Mitchell (1818–1889), first female professor of astronomy at Vasser and first woman in the National Academy of Arts and Sciences, an unfortunate record that lasted until 1943

—

"There is an infinitude of things you don't know, but it's not an interesting infinitude, because it has no grain. Only some of the unkown things, a much smaller subset, are things that you are aware of not knowing, and then within that subset is a smaller set still—the unknowns that pull at you, that seem meant for you alone to know. Curiosity is a way of ordering and indeed paring down the wildness of the world. Of all the unmowed fields, this one right here is the one I would like to make mine. Why? Because it happens to be here and because I will contribute most efficiently to the whole if I study it, knowing that it is obscure enough that nobody else is likely to want to spend much time on it."—Nicholson Baker, Mowing

—

“The failure of modern science to incorporate this material ‘I’—this ‘self’, this ‘mind’, this ‘spirit’, this ‘soul’—into its world picture is one of the premier pathologies of modern Western culture and, sadly, one reason why many people are now turning away from science.”—Margaret Wertheim, The Pearly Gates of Cyberspace

—

“As an adolescent I aspired to lasting fame, I craved factual certainty, and I thirsted for a meaningful vision of human life—so I became a scientist. This is like becoming an archbishop so you can meet girls.”—Matt Cartmill, anthropology professor and author (1943–)

Contact and copyright information

The Exchange is published on behalf of the Scientific Communication special interest group of the Society for Technical Communication. Material in the Exchange can be reprinted without permission if credit is given to the author and a copy of the reprint is sent to the editor. Please send comments, letters, and articles to the editor.

Editor, publisher, and SIG manager:

Geoff Hart (ghart@videotron.ca)

Webmaster:

Geoff Hart (ghart@videotron.ca)

© 2004, Society for Technical Communication (901 North Stuart St., Suite 904, Arlington, Virginia 22203-1822 U.S.A., 703-522-4114, 703-522-2075 fax, www.stc.org)