Docket Management Facility
US Department of Transportation
400 Seventh Street, SW
Nassif Building, PL-401
Washington, DC 20590-0001
Re: Federal Transit Administration Docket Number 2006-23636
(Download in Adobe PDF format)
On behalf of the more than 1,600 member organizations of the American Public Transportation Association (APTA), I write to provide comment on the Federal Transit Administration's (FTA) Notice of Availability of Guidance and Request for Comments concerning New Starts Policies and Procedures, published January 19, 2006, at 71 FR 3149. We respond below to the issues raised in the order in which they appear in the Notice.
About APTA
APTA is a non-profit international trade association of more than 1,600 public and private member organizations, including transit systems; planning, design, construction and finance firms; product and service providers; academic institutions; and state associations and departments of transportation. More than ninety percent of Americans who use public transportation are served by APTA member transit systems.
General Comments
APTA greatly appreciates the extensive efforts FTA staff has made to capture their preliminary thinking about changes to the entire New Starts evaluation and rating process, as well as the opportunity to collaborate with FTA as it completes its analysis of these and other proposed changes to the program.
The Need for Rulemaking Processes to Implement Significant Changes
APTA strongly believes that certain proposals contained in Part 1 are of such a magnitude that they are inappropriate for implementation in the fiscal year 2008 submittals. Our objections and concerns, described below, apply whether these changes are implemented in the current submittals or in the permanent changes to the program, but are elevated by the prospect of immediate implementation. These changes should be considered only, if at all, in the Part 2, formal rulemaking process when FTA issues a Notice of Proposed Rule Making (NPRM). These proposals, including National Environmental Policy Act (NEPA) Interfaces, Preservation of Information for Before and After Studies, Project Development Agreements, and New Starts Funding Level Set at Final Design Approval should be deleted from the guidance and conditions left in place as currently defined by FTA until after they have been the subject of a NPRM. The reasons for this are discussed in detail in the comments that follow.
Working Session Recommendation
In order to strengthen the understanding of both APTA and FTA of the full implications of the proposals contained in Part 2, which will be the subject of a NPRM later this year, APTA requests that FTA give serious consideration to some other mechanism to enable a productive, two-way conversation between FTA and expert representatives of the transit industry about some options for improving the entire New Starts evaluation and rating process. A 'shirt-sleeves' working session prior to drafting of the NPRM would allow FTA to confer with knowledgeable members of the transit industry, could constructively pursue many of the concerns discussed in the comments that follow, and help hammer out some ideas to solve the very real problems faced by FTA in rating New Starts projects.
Total Effect of the Changes Proposed: Increased Burdens
The proposed changes will add to the burdens of the program, both for FTA and for project sponsors, without a commensurate improvement in information for decision-making. Cumulatively, these changes and those made in recent years will increase the cost of planning and project development, will extend the time required for good projects to proceed to implementation (and thus increase implementation costs), and will add to the uncertainty about project approvals that transit agencies face. These additional costs will largely be imposed upon local transit agencies since Alternatives Analysis (AA) is now ineligible for section 5309 funding and FTA's proposed capping of the section 5309 funding share at the start of Final Design. This will make transit investments less attractive than highways in the eyes of many local officials, since the highway planning and project development process is less burdensome, and the Federal Highway Administration (FHWA) will participate in all costs. APTA and the transit industry believe a 'level playing field' between highway and transit project consideration is critical.
Many of our members find themselves frustrated with the current New Starts evaluation process, and seek ways to streamline that process, rather than add new burdens. Although we clearly recognize FTA's duty to credibly rate the many New Starts project proposals, we believe there is a more effective means of doing so.
Risk and Uncertainty
FTA seems to be trying the difficult - if not impossible - task of eliminating all risk and uncertainty from forecasts that are, by their nature, somewhat risky and uncertain. FTA and the transit industry have come a long way over the last 15 years to increase the reliability of cost estimates and travel forecasts. The Safe, Accountable, Flexible, Efficient Transportation Equity Act - A Legacy for Users (SAFETEA-LU) requires FTA to explicitly consider the reliability of cost estimates and user benefit estimates, but the statute clearly does not require a new and costly overlay of process and analysis in an effort to eliminate risk. Current best practices for Quality Assurance and Quality Control are effective, and give FTA an adequate basis to meet new SAFETEA-LU requirements.
Evaluation Framework
In short, APTA believes that a "new paradigm" is needed for FTA's management of the New Starts program and the evaluation and rating of projects. The new paradigm would simplify the New Starts process and reduce the time and cost of planning and project development. Some possible concepts might include earlier screening out of projects that are not worthy of extended effort or making preliminary funding commitments earlier, based on planning and alternatives analysis, to reduce the uncertainty that transit agencies face when they decide whether to expend resources on Preliminary Engineering (PE).
General Comments on Part 1
Part 1 is intended to include proposals that could be implemented through FTA guidance for the FY 2008 New Starts Report. APTA believes that some of the proposals in Part 1 are not appropriate for this guidance document - some ought to be made only through regulation, while other proposals should be included in separate guidance documents (i.e., Procedures and Technical Methods). Any proposals that change the fundamental structure of the process, including required activities in each phase, and funding eligibility requirements, should only be enacted by regulation, using a Notice of Proposed Rule Making.
Specifically, APTA believes that proposals that should be made by regulation include NEPA Interfaces, Preservation of Information for Before and After Studies, Project Development Agreements, and New Starts Funding Level Set at Final Design Approval.
Proposals that should be included in other guidance documents include guidance in the conduct of Alternatives Analysis, and Development of Costs and Ridership Forecasts. In the meantime, the procedures that are the subjects of those proposals should be left in place as they now are being applied.
On page 4, the proposed Guidance document says that FTA "requires" project sponsors to submit certain study products as they are developed during AA. If these are truly required, they should be developed through the notice and comment rulemaking process. If they are "useful," as FTA has said in their recent AA training, they should be covered in the AA guidance. The New Starts Reporting Guidance seems to be the wrong place to publish this list.
APTA is also concerned about two areas that Part 1 did not include:
- Changes in Transportation System User Benefit (TSUB) thresholds reflecting inflation, as was promised in the Dear Colleague letter of April 2005. This is also discussed in some detail in the APTA response to Part 2.
- Discussion of the current FTA practice of limiting changes in costs from the estimates contained in the Full Funding Grant Agreement (FFGA) to 5 percent even when all of the additional costs are paid from local sources.
Specific Comments on Part 1
Section 1
Our concerns about the proposals for the AA process include the following:
- The "required" portions of this guidance, if further considered at all, should be the subject of rulemaking.
- The new requirements for FTA review and approval of AA methods and results, prior to approval for the project sponsor to enter PE, adds another step to an already lengthy, complex and linear process, increasing total time for project development, with no perceived benefit or certainty for project sponsors.
- There is great concern based on past experience with FTA response time, that because of inadequate staff resources, FTA will not be able to make timely review and responses to the proposed new multiple steps during AA, resulting in lengthy waits for approvals to proceed. This adds to project sponsor consultant costs for AA, substantially delays planning work, and makes it difficult to achieve buy-in and understanding of the planning by local elected officials and the public.
- As FTA has acknowledged, estimates are expected to change as engineering progresses. For this reason, it appears to APTA that there will be limited value in extensive efforts to evaluate changes in cost estimates between AA and the FFGA.
Section 1.1, NEPA Interfaces
APTA strongly believes these proposals should be accomplished, if at all, by rule-making, not by New Starts reporting guidance.
Requiring scoping during AA seems to be moving the process in a different direction than is FHWA in implementing Section 6002 of SAFETEA-LU. FHWA is saying that Project Initiation under SAFETEA-LU occurs only when there is "a project." In FTA terms, that means after there is a Locally Approved Alternative. FHWA is further saying that the Notice of Intent (NOI) and Scoping should occur after Project Initiation. A consistent interpretation of Section 6002 by both FHWA and FTA would be helpful and is absolutely essential in the case of multi-modal corridor studies and projects. NEPA itself encourages multi-modal corridor studies, and FTA has done so in the past, but this lack of consistency will act as a major constraint on joint highway/transit projects.
APTA is strongly opposed to the proposal to not issue a Record of Decision (ROD) on an EIS unless FTA has made a determination that the project has achieved a New Starts rating of medium or higher. APTA fully understands the problems that FTA is trying to address in these proposals, including the desire to include all information relevant to decision-making in NEPA documents, and the desire to eliminate needless production of supplemental NEPA documents after a ROD has been approved. APTA supports these goals, but we believe that there are some better ways of accomplishing them, as suggested below. In any event, changes should only be implemented by rulemaking, not by guidance. Regardless of the effect on efficiency, there is a significant chance that requiring an acceptable New Starts rating prior to signing NEPA documents may actually compromise the environmental review process and subject projects to legal attack based on the implication that the NEPA result was preordained in the New Starts process. This may jeopardize projects through delay and litigation.
FTA has a tradition of success in streamlining the environmental process while fulfilling its environmental stewardship responsibilities under NEPA. This success has been evidenced by the shorter timeframe to complete the EIS process for major transit projects in comparison to highway projects (an average of 3.5 years for transit versus 5 years for highways). APTA recognizes and applauds FTA's commitment to sharing New Starts information with the public, and realizes that the NEPA process affords FTA the opportunity to achieve this important objective. However, it is APTA's position that this disclosure can be accomplished in a manner that does not cause unnecessary delay to the NEPA and New Starts process.
In light of these concerns, we propose the following alternative approach:
During the FEIS preparation, the New Starts information available at the time of FEIS publication should be incorporated into the document, so that FTA can disclose to the public in the document the current status of the project in the New Starts pipeline, as well as the subsequent steps necessary to advance the project through the New Starts process.
With respect to the ROD, APTA recommends that FTA incorporate a New Starts Finding into the ROD, in a manner similar to other findings that support the closure of the NEPA process under Section 4(f) and DOT Orders, such as the protection of floodplains and wetlands. In this finding, FTA can recap the status of the financial decision under New Starts, and disclose the project rating. For projects that have achieved a "Medium" rating, FTA can disclose to the public that if the project will undergo any modification subsequent to the ROD, then modification of the ROD may also be required under any subsequent NEPA reviews, as appropriate.
APTA believes that the alternative approach described above would support FTA disclosure objectives, while allowing continued public discussion of the project while it progresses through the project development process. Further, it would help achieve another important aim for both FTA and the project sponsor - to reduce project risk. Risk would be reduced in at least three ways:
- The ROD starts the legal clock ticking for the 180 days to appeal the finding, enabling that legal risk to be dealt with sooner;
- The issuance of the ROD while the FEIS information is current avoids the risk of the information becoming stale during delay, and the need for costly updates; and
- The ROD allows project sponsors to begin critical ROW acquisitions, to avoid local problems resulting from delayed acquisition, and associated property cost increases.
Section 1.2, Preservation of Information for Before and After Study
APTA believes this proposal should be the subject of rulemaking and is inappropriate for inclusion in the current year reporting guidance.
Section 1.3, Certification of Technical Methods, Planning Assumptions and Project Development Procedures
APTA strongly opposes the proposed requirements for certification. APTA believes FTA can ensure the competence of work products and meet its obligations under SAFETEA-LU without certification, that certification adds little, if any, certainty to that process, and that this proposal exceeds the intent of Congress in SAFETEA-LU or Title 49, United States Code.
FTA is entitled to guard against deliberate distortions of data, or the direction of work on New Starts projects by totally unqualified persons. FTA can accomplish this through continuing review of methodologies employed and conclusions reached within the Project Management Oversight (PMO) program. Certification would be unlikely to remedy these perceived problems.
Project sponsors make every effort to follow FTA direction, but often the guidance is unclear - as in the definition of the Baseline Alternative - or is still in flux, waiting new instructions from FTA. Who can certify that they are following FTA procedures, if the FTA procedures are not clear?
It is not appropriate to have a consultant sign certifications. Local project managers are responsible for what their consultants do, and ensuring that they follow FTA guidance. Many aspects of the analysis are outside of consultants' control, such as population and employment forecasts, or are imposed by the client. The project sponsor has the ultimate responsibility, and should assume it.
The word "certification" could be taken to mean an essential guarantee of the work in question. The DPIC Contract Guide states, "By definition, the words certify, warrant or guarantee mean to assure the total accuracy of something or to confirm absolute compliance with a standard. Legally, these words and their derivatives are virtually synonymous. Therefore, if you certify or warrant something, you are guaranteeing that something is unequivocally true or correct or perfect." The proposed requirement therefore sets an unreasonable standard of care threshold for consultants, raising liability concerns, with the possible result of voiding existing errors and omissions insurance. It further raises many concerns about the assignment of risk among parties, and who should indemnify a contractor asked to certify results.
Finally, APTA support the comments submitted by the American Council of Engineering Companies (ACEC) in their response which provides that the proposal exceeds Congressional intent, creates a certification based on uncertain standards and conventions, fails to account for fluidity of responsibilities within project development, sets an unreasonable standard of care, threatens to void errors and omissions insurance, creates an uncertain enforcement situation, fails to account for the responsibility of data sources, and fails to provide a means wherein consultants may review and comment on reports concerning their performance.
Section 1.4, Development of Costs and Ridership Forecasts
Including appropriate information about uncertainty with any forecast is really good practice, and has been advocated in FTA guidance for many years. While APTA agrees that it is good practice to define uncertainty, it is not clear how comparing cost and ridership estimates at four points in time will help either FTA or project sponsors to better understand those uncertainties, which are based upon numerous and complex factors that vary across agencies, regions of the country and methodological frameworks and vary for the same project over time.
The proposal does not describe how the uncertainty analyses will be used by FTA in its evaluation of projects - whether probabilities be added to the range of estimates, and to cost-effectiveness thresholds.
The FTA focus on a single number, whether the Cost Effectiveness Index (CEI) or TSUB, with a minimum threshold requirement, tends to make it more difficult for project sponsors to fairly address uncertainty analysis.
In dealing with the development of costs, FTA should also consider the findings of its commissioned study, "Analysis of Capital Cost Elements and Their Effect on Operating Costs," which cites institutional capacity, level of in-house expertise, regulatory mandates, level of competition among vendors, and the use of custom designs as important indicators of the probability of cost increases.
Section 1.5, Project Development Agreements
The proposals in this section should be deferred and accomplished by rulemaking, not by guidance.
This concept has merit but succeeds or fails on FTA's partnership with the project sponsor to move the project through the development process, and assurance that it will meet reasonable deadlines in return for project sponsor agreement on local actions. It may also offer special opportunities to apply innovative implementation methods, such as design-build or Construction Manager/General Contractor (CM/GC). However, the agreements must protect agencies against changes in processes or requirements during the course of the project, unless specifically mandated by statute.
We are concerned that FTA may simply use the agreements as a method to remove projects from the New Starts pipeline, and are further concerned over repeated use of the phrase, "FTA, at its discretion…." This could result in inequitable treatment of some projects against all others. Who will be selected and on what basis? Will project sponsors have any say in whether they are selected?
If regulation is required to make project development agreements universal, as indicated in the last paragraph of this section, should not rulemaking also be required for FTA to implement them selectively?
Section 1.6, New Starts Funding Level Set at Final Design Approval
The proposal contained in this section should be accomplished, if at all, by rulemaking, not by guidance.
This proposal changes the fundamental definition of Preliminary Engineering (PE), by extending PE to include design and cost estimating activities that traditionally have been accomplished as part of Final Design (FD). As such, it should only be accomplished by rulemaking.
The artificial boundary between PE and FD may have outlived its usefulness, along with the "check-in and check-out" requirements that FTA now imposes for each of these phases, and this could be the beginning of FTA recognition of that fact.
This proposal could have unintended adverse consequences, by limiting the options for unconventional implementation (design-build, CM/GC, public-private partnerships) versus conventional design-bid-build. Alternatively, we believe it could have the effect of making design-build or CM/GC essential for providing cost assurance to both local agencies and the FTA, and making it possible for the funding cap to be supported by a firm, fixed price and bid bond from a contractor.
There will be a major impact on the cost of PE (because of the level of design necessary to set funding caps). The 8 percent limit on 5309 funding for PE could result in pushing the added burden of those costs onto local agencies.
This proposal could affect the timing of the entire project development process, and thus could affect the timing of projects qualifying for federal reimbursement for ROW acquisition and other activities that can only occur after a ROD. Any delay in project development also results in escalation-based cost increases.
Projects should not necessarily be removed from the New Starts pipeline (under a PDA) if additional engineering during Final Design results in cost increases. Instead, the project sponsor should be responsible for covering the full amount of any cost refinements after the funding cap has been set.
Section 1.7, Possible Rules for Mode-Specific Constants
This proposal addresses a real problem and suggests some solutions that do not require time-consuming research by the project sponsor. While it will require the recalibration of some constants in the mode-choice model, the effort required will be only a few weeks if the existing travel model is in good shape. FTA should be applauded for crafting a workable solution to the problem.
The proposed guidance, however, gives very little information on the specific procedures to be followed to modify the models; on the analysis that leads to the conclusion that 12 minutes is an appropriate value for the rail constant; or on what review requirements may be imposed on the model stream before FTA approval will be given.
An approach that responds to the problem, while the above questions are being answered would be to use either Option 1 or Option 2 (shown on page 13 of the Guidance document) on a trial basis for the FY 2008 New Starts Report submittal in those regions that are proposing a new mode. At the same time, FTA and APTA could establish a small panel of experienced practitioners, charged with preparing a working paper describing the evidence for a specific value or values for the constant. This panel would review the experience of those who used the constant in their FY 2008 submittals, and would also gather input from the transit community at large. The working paper would thus provide some experience on which to base permanent guidance.
We would be much more comfortable with the use of a constant representing 12 minutes of in-vehicle time if we had additional information on how that number was derived. Specifics that would be helpful include: 1) which models were reviewed, 2) whether they included heavy rail, light rail and commuter rail systems, 3) whether they were recently calibrated, 4) whether the models were all structurally similar, and 5) whether cities from different geographical regions were represented. We question whether using 15 minutes rather than 12 minutes would result in higher ridership estimates and greater user benefits. The recent success of rail systems across the country, and the response to rising fuel prices would suggest that a more generous constant might be justified. We would also like to understand the constraints in how the model was applied. Ideally, some real-world case studies would answer a lot of questions.
Option 3 (on page 13 of the Guidance document), varies the constant within limits, based on project characteristics, and could ultimately have an advantage over a one-size-fits-all approach. However, FTA should engage the industry in discussions of the project characteristics to be considered before this option is used as the basis for an ultimate recommendation. Some project characteristics can clearly be quantified, such as span of service, frequency, or degree of grade separation. But other project characteristics, such as visibility, are subjective.
The Guidance document raises the issue of alternative specific constants being used as a correction factor for shortcomings in the models. Recent experiences have made it clear that FTA has some concerns with the trip distribution and mode choice models. It would be very helpful if those concerns could be articulated before travel projections are made, rather than being discovered as part of an FTA review of project submittals. Given the critical importance of the results of travel forecasting, a set of standards, or a checklist of necessary model characteristics is a necessity.
In summary, APTA believes that FTA has suggested a workable solution to a clear inequity in the current process. However, we feel a need for more background on the values being suggested before we can draw hard conclusions. The interim approach suggested here, along with a commitment to answer some of the questions surrounding the Guidance proposal, would allow the inequity in the current policy to be remedied while the need for further information is being met.
In addition to the interim approach suggested above, there is concern among some transit agencies that it is not realistic for FTA to release these details at the end of April, and expect them to be folded into the analyses in time for the June submittal of ridership data for the next New Starts report. The constants will affect ridership forecasts, and thus the operating plans, requiring new model runs for equilibration, changes to capital and O&M cost estimates, CE calculations, financial plans, etc. However, if some project sponsors are sufficiently prepared to do so, they should be able to do so as an option rather than as a requirement.
While establishing limits to the benefits of introducing new guideway modes is important for some regions, in other regions with mature transit systems, the constants should be carefully calibrated to match existing conditions.
General Comments on Part 2
As stated at the beginning, APTA appreciates the effort FTA has made to provide the transit industry in Part 2 of the Guidance document with what is, in effect, an Advanced NPRM, giving us a chance to respond to changes that will be proposed in a formal NPRM next September. FTA has proposed a number of ideas in Part 2 that we believe have merit, but are unfortunately proposed essentially as add-ons to existing New Starts requirements, rather than as fundamental changes in the entire process. We therefore propose an effort by FTA and APTA this spring to collaborate on a more fundamental change in the evaluation and rating process that we believe to be timely and desirable.
Our concerns about Part 2 fall into two general categories:
There is concern that the intent of Congress and statutory language contained in SAFETEA-LU may have been inadequately expressed or represented in some of the proposals contained in part 2.
There is concern that the cumulative effect of the proposed changes in Part 2 will be to add to, rather than to streamline the project evaluation and rating process, making the project development process more burdensome, both for FTA and for project sponsors, and greatly increasing the risk to project sponsors. It would be ironic indeed if the very efforts of FTA to better quantify and deal with project risk resulted in a substantial increase in project risk for transit agencies. FTA wants more accountability, and rightly so, but that may be better achieved by partnership, rather than what is perceived by some to be an adversarial approach.
Specific Comments on Part 2
Section 2.1, Eligibility and Definition of Fixed Guideway
FTA should not address the definition of Fixed Guideway in the regulations. The definition now in the law has worked well over the years, and allows sufficient flexibility. If FTA feels that it will be beneficial to specify how Bus Rapid Transit (BRT) will be defined, or whether FTA should participate in High Occupancy Vehicle (HOV) projects, this could be done in guidance, but should not be made unduly rigid.
Any BRT project that is funded with New Starts funding should have a fixed guideway component that is consistent with existing law.
While APTA recognizes the value of HOV projects for transit service, HOV projects that allow carpools to mix with transit vehicles should not generally be eligible for New Starts funds. Existing law provides for such a determination.
Section 2.2.0, Evaluation Framework
The Guidance document proposes two alternative frameworks for organizing and rating New Starts evaluation measures: 1) an extension of the existing framework, but with new measures required by SAFETEA-LU added; and 2) development of a broader framework, briefly outlined in the Guidance document.
APTA does not support either Option 1 or Option 2, but suggests FTA instead adopt a new paradigm for evaluation and rating of projects - Option 3.
Option 1 continues the existing practice of defining cost-effectiveness only in terms of mobility, and only transit mobility at that. SAFETEA-LU requires changes in the criteria to give land use and economic development equal status with mobility improvements. In response, FTA should broaden its definition of cost-effectiveness to include land use and economic development benefits, as well as mobility benefits accruing to highway users.
The 'broader framework' proposed in Option 2 continues the past practice of defining cost-effectiveness solely in terms of mobility for transit users. Further, it does not even acknowledge land use as a benefit, but relegates it to a "project uncertainty" indicator. This seems to be inconsistent with the language of SAFETEA-LU, which recognizes land use and economic development as potential benefits of New Starts projects.
Option 3 - A New Paradigm. Substantial effort will be required by FTA and the transit industry to develop an Option 3 that will resolve the issues with Options 1 and 2 outlined above. However, APTA is prepared to work with FTA before the NPRM is drafted to develop a new framework that is technically defensible and that will allow agencies to draw upon current analytical tools and data sources. The Guidance document itself, in Section 2.2.1.2, broaches the possibility of broadening the definition of cost-effectiveness by including economic development, an encouraging initial step.
Analytical perfection should not be the goal. The cost-effectiveness measure needs to be sufficient to identify the best projects, the worst projects, and those in the middle. If this is the approach, the challenges of trying to avoid double-counting, of dealing with the differences between transfers and net increases in regional economic terms, and the difficulty of combining dissimilar benefits, all become less daunting. A point system, similar to the one proposed in the Transit Cooperative Research Program (TCRP) Quick Response Project J-06/Task 66 Small Starts Justification Criteria, a copy of which is enclosed, might offer a useful point of departure for discussion.
Further, FTA should recognize that not everything is quantifiable. Reducing the evaluation process to a series of quantitative measures, weights and risk factors misses the need for judgment about some factors that are important, yet inherently subjective in evaluation of projects.
An important advantage of such a new paradigm to both FTA and the industry could be the early self-screening out of projects that are not worthy of extended effort. Currently, the work required just to make a very preliminary determination of potential scores of cost-effectiveness is so burdensome that it is difficult for project sponsors to judge how well the project might score nationally. As a result, the very time and effort involved in determining how good the project might be have the effect of building up momentum and support for some projects that realistically should not be pursued.
FTA efforts to respond to SAFETEA-LU requirements for certainty analyses in forecasts has been to mix risk and uncertainty together with cost-effectiveness, land use and other factors, an approach that does not support either proposed evaluation framework. We believe that FTA has incorrectly characterized the language and intent of the statute on uncertainty.
There are sufficient analytical tools available to make reasoned choices, but to efficiently employ these tools, the decision process must be simple, inexpensive, and fast. FTA should work with the industry to come up with a new system that works efficiently and is understandable to our constituencies.
Section 2.2.1, Specific Evaluation Measures
The language of the FTA Guidance document suggest that FTA does not believe that land use changes are project benefits - they simply represent an environment that may reduce uncertainty about ridership. It suggests that only mobility benefits are important, notwithstanding statutory language requiring equal consideration of land use and economic development.
Lacking a uniform method of quantifying land use and economic development benefits, FTA should allow for the importance of these measures recognized by Congressional intent in a less quantified way. Actual commitments and adoptions of land use actions to increase densities around stations by local agencies should be recognized as valid measures of change that will benefit transit.
Any measure of land use and economic development should also consider the benefit of maintaining existing land uses and economic activities in mature cities.
Concerns about economic transfers versus net increases, should be approached in a more subjective way. In fact, most regions clearly see that certain changes in land use and economic development within a region - transfers - are desirable, and a real benefit that cannot be overlooked. Such transfers can help reduce sprawl and auto commuting, improve transit usage patterns, and improve productivity of some under-utilized land areas within the urbanized region.
Mobility benefits should measure more than just transit user benefits. Last year, FTA worked on a methodology for measuring benefits to highway users as well and that work should be pursued and included by FTA.
Better methods for handling non-home-based trips would enhance the quality of the forecasts.
FTA proposes to remove operating efficiency as a separate evaluation criterion, on the grounds that it already comes into play in the calculation of cost-effectiveness. We disagree with this proposal since operating efficiency can be a significant factor in comparing a single new rail line with the transit system as a whole, as has been shown in Honolulu and other places. Even if operating efficiency is removed as a separate evaluation criterion in effectiveness, it should still be used in the financial sufficiency analysis, where it can be very important.
Section 2.2.1.2, Cost-Effectiveness
APTA recommends adjusting the CEI thresholds by at least the Consumer Price Index (CPI) percentage annually.
APTA commissioned a task force to study cost effectiveness indices and we have attached a copy of their report to these comments. We have provided a brief review of our findings in the following paragraphs.
In its evaluation of cost-effectiveness of a proposed project, FTA considers the incremental cost per hour of transportation system user benefits in the forecast year. This measure, expressed in constant base-year dollars, is based on the annualized total capital and annual operating costs, divided by the forecast change in annual user benefits, comparing the proposed project to the New Starts baseline alternative. The current CE thresholds used by FTA for assigning High, Medium-High, Medium, Medium-Low and Low cost-effectiveness ratings were set in the Dear Colleague letter of last April.
FTA has proposed adjusting these threshold figures in future years by applying an adjustment factor based on changes in the Gross Domestic Product (GDP). The problem of the proposed approach is that the cost of construction, which is a major component of the annualized capital cost, has recently been rising (from 2002-2005) at a much higher rate than the rate of growth of the GDP. We analyzed the trends in GDP, the CPI and the Engineering News Record (ENR) Materials Price and Skilled Labor Indices (ENR) with a curve representing 70 percent of the composite construction cost increases from 2002 through 2005. (The reason we use 70 percent of the construction cost growth rate is that this is roughly the percentage contribution of annualized construction costs to the total of annualized capital costs and annual Operations and Maintenance (O&M) costs in the CE calculations of most New Starts projects. Thus, the use of 70 percent ignores any growth in annual O&M costs over the period.).
Between 2002 and 2005, construction costs (ENR) rose by 21.92 percent, whereas the GDP rose by only 4.43 percent, and the CPI increased by 15.4 percent. Assuming annualized construction costs contribute 70 percent of the total of annualized capital and annual O&M costs, then 0.7 percent of the ENR increase equates to 15.34 percent during this three-year period, which is virtually the same as the growth in the CPI. So adjusting the Cost-Effectiveness thresholds by the rate of growth of the CPI appears to be a more appropriate adjustment factor than the GDP.
An analysis of current New Starts projects was made to determine how each project would fare, with threshold adjustments. If the current $21.99 upper limit for a Medium rating is not adjusted at all through 2009, and recent construction cost trends continue, all but three projects would become ineligible. If the CE Medium threshold figure is adjusted by the current annual growth in GDP, and current construction cost trends continue, then all but five of these projects will become ineligible by 2009. However, if the CE Medium threshold figure is adjusted to the CPI rate of growth, and current construction cost trends continue, then all of the currently eligible projects will still be eligible in 2009.
Additionally, we believe the issue of the new CE thresholds set by FTA last April must be addressed. There is general confusion among project sponsors concerning the reasons for the changes and those changes have proven quite disruptive.
The threshold issue is closely related to the issue of definition of the baseline alternative.
The inclusion of risk and uncertainty analyses in cost-effectiveness suggests that FTA is not only adding to the time required to make project evaluations, but is foisting all risk on the project sponsor.
There is a strong minority view that O&M costs should not be included in cost-effectiveness calculations at all, because they overweight annualized capital costs, and because they are costs paid only by local agencies, not by FTA.
We believe FTA should provide grantees with linked spreadsheets for all New Starts calculations, including Summit outputs, rather than the manual templates now required. The linked spreadsheets should be similar to the Standard Cost Category spreadsheet, and allow automation of the templates. This would simplify much of the current quality control checking process developed by FTA, would simplify the New Starts reporting process for both grantees and FTA, and would provide an additional level of transparency to the process.
Section 2.2.1.3, Financial Capability
It is unclear from the Guidance who should be responsible for preparing the assessment of the reliability of financial forecasts - the project sponsor, the FTA consultants who review financial capability - or both.
The FTA proposal (page 28) appears internally inconsistent concerning overmatch, saying that FTA will not discriminate against project sponsors who do not overmatch, but, in the next sentence, that high funding share may result in a higher project rating.
The statute allows FTA to use overmatch as a rating criterion, but only if they also consider the underlying economic capability of the community. FTA has two choices under the statute - they can either not consider overmatch at all or also consider the agency's financial capacity. The discussion in the Guidance document does not make this clear, nor does it make clear whether FTA contemplates that overmatch should be or will be included.
FTA proposes to continue the current weighting of grantee's financial capability for project capital funds at 50 percent; for project O&M funds at 30 percent; and for other non-project funds at 20 percent. This is awkward, and does not yield a meaningful result. The capital funding plan should simply be pass-fail. The O&M funding plan should be considered in a less mechanical, more subjective manner.
The reliability of proposed project funding sources should depend upon the stage of the project. It's far less important that a proposed funding source be actually in place at the AA stage, or even in PE, than it is at the FD stage. But FTA does a risk analysis at the outset now.
The checklists for financial analysis provided in the Annual New Starts Report are helpful and make the analysis process more transparent than it was in the past. However, there still seems to be some variability among the FTA financial consultants on how they interpret data against these checklists. More standardization is necessary for full transparency. FTA should provide more information about how the charts are used as a basis for ratings and take positive steps to ensure consistency among its contractors.
Project capital and O&M costs get covered in the justification criteria, and then get covered again in the proposed "reliability of forecasts." This appears to be "double counting," as described more fully below, and should be addressed in detail.
SAFETEA-LU requires that the reliability of cost estimates be included in the cost-effectiveness review, but not in the financial analysis. It confuses the evaluation if reliability of cost estimates is included in both places. The financial analysis charts and criteria in the New Starts Report seem to apply only to sources, not to costs.
The proposed requirement for letters of endorsement from local agencies on the financial plan during PE is premature.
Section 2.2.1.4, Reliability of Forecasts
The accumulation and layering of risk analyses in the Guidance document suggests "double counting." The Congressional intent was to evaluate the reliability of cost estimates and ridership forecasts, but that intent is expanded in the proposal to include looking at many other factors as well. The proposal is confusing, and based upon recent experience with project risk analyses, we are concerned that the proposal includes substantial effort with little reduction in uncertainty. The best way to reduce uncertainty for both FTA and project sponsors is to simplify and streamline the process
Regardless of the efforts of project sponsors on risk and reliability of forecasts, sponsors still cannot predict the outcome of FTA's risk assessment. The revised evaluation framework proposed in the Guidance document calls for even more risk analyses. These lengthy reviews actually add cost escalation risk and are not in the interest of FTA or of project sponsors.
Forecasts have improved substantially in recent years. Requirements for more risk analyses seem a surrogate for quality control in methodology, but adequate quality control already exists. FTA should clearly distinguish quality control from reliability of forecasts. FTA already encourages project sponsors to use a number of mechanisms to improve both quality control and reliability of forecasts, including peer reviews of cost estimates and FTA review and approval of ridership forecasting models and outputs. Those efforts are more valuable than additional risk analyses.
FTA suggests higher contingencies if reliability of cost estimates is not deemed sufficient, but gives no indication of how high those contingencies might have to be. Higher contingencies not only increase the total project costs, they also affect the CE or TSUB numbers. Instead, FTA should provide incentives to project sponsors to control costs.
FTA should recognize that transit agencies often do not control land use or create land use projections, and thus cannot assure land use data reliability. A better approach would be to oversee reliability through FTA joint efforts with FHWA in certifying Metropolitan Planning Organizations' products.
Section 2.3, Project Development Procedures
FTA should avoid adding new steps and endorsements unless clearly justified or required by statute. The existing process is recognized as rigorous and should be streamlined. New requirements should be considered only if there is a widespread problem, and the proposed solution is worth the added cost. Every project sponsor should not be made to go through new process steps to address problems that have only cropped up in isolated cases. The excellent questions asked by FTA in Guidance Section 2.2, on the top of page 20, should be applied to these proposed changes in project development procedures. FTA should "make the case" for these and other proposed changes.
Regarding the proposal to require local endorsement of the financial plan, APTA feels that it is unnecessary and burdensome to the planning process, particularly at the PE level. It will be difficult to get financial commitments from local government without a corresponding commitment at the same time from FTA. There is some concern that the requirement could actually give local governments a "hold" over projects and result in pressure for project betterments and thus increase project costs. FHWA does not require a similar financial endorsement from state and local governments before highway projects move along.
With respect to the proposed requirements for review and approval of the Baseline Alternative, APTA is strongly concerned that the entire area of definition of a Baseline Alternative is a major problem, and the proposed changes do not solve it. FTA guidance in the past has not been sufficiently clear and the requirement has not been consistently applied, casting doubt on the entire project rating system. A particular challenge is the need for the baseline to be both "consistent with the Build Alternative," and "optimized as the best bus alternative." Each project is unique, and requires negotiation/discussion between FTA and the project sponsor, sometimes over many - even dozens - of draft alternatives. As a result, many transit agencies feel that it would be better to use the No Build as the baseline. We feel that clearer guidance is needed of how to balance "consistency" with "optimization," possibly including examples.
We are concerned about the proposed requirement for an On-Board Transit Survey every five years. Rather than a blanket requirement, we suggest that part of the QA/QC for travel models should consider backup data, including the availability of reasonably current on-board transit survey information. On-Board travel surveys are not the sole means of obtaining information on travel patterns. Electronic fare collection systems can provide extensive data about travel patterns far more reliably and cost-effectively, particularly in large transit systems. In some cases, a transit agency may clearly need a new on-board survey, but in other cases it may not, or the need can be met by a less expensive "mini-update" or sampling procedure. It should be judged as one of several factors in overall model quality control.
The proposed requirement for Project Reaffirmation by the MPO at the time of entry into Final Design, although seemingly benign, adds an additional step to the FD approval process. Agencies work closely with their MPO already and projects are always included in the appropriate TIP. Plans are regularly updated and MPOs reaffirm projects in each update already. Including an additional reaffirmation will simply add time to project schedules. It would not add a new level of certainty or provide any special benefit. Further, any such FTA requirement should be made consistent with FHWA treatment of highway projects in the region.
The possible implementation of New Starts Funding Share Incentives as provided for in SAFETEA-LU is viewed with some skepticism by most of our members. The money to provide such incentives would be better spent on good projects. An alternative incentive plan would allow project sponsors who successfully control costs and bring projects in under the FFGA amount to attribute the unused federal funds to other projects or betterments of the current project. Many of our member transit agencies feel that project sponsors do not need additional financial incentives to achieve good projects, because the local agency carries most of the risks from cost overruns and poor ridership results anyhow.
Again, APTA appreciates the FTA effort to provide advance information about proposed changes. The comments above provide some background on transit industry concerns with the specific proposals in the guidance document. However, of most importance, we believe that now is the time to seek a "new paradigm," rather than proceed with continuing additions to an already lengthy, complex and often cumbersome process. We feel that such a new paradigm will not only be of value to transit agencies, but will relieve FTA of an increasingly heavy burden.
Sincerely,
William Millar
President and CEO
American Public Transportation Association
Attachment:
Report of Task Force on Cost-Effectiveness Thresholds
