The plan was unveiled on Wednesday by the European Commission, the EUs executive arm, and would require the approval of the European Parliament.
EU member states, which make up the European Council, have made a similar proposal, but it would require data to be stored for a longer period and would force service providers to bear the full costs of compliance—estimated at nearly 200 million euros per company per year.
Both proposals would require compliance from service providers located outside the EU if they carry traffic to the area.
Storage costs are set to be a major issue in the debate over the proposals by the member states and the Commission.
A report by the European Parliament on the proposal put forward by the UK, Ireland, France and Sweden earlier this year criticized the plan as impractical, and said it would place an immense financial burden on Europes telecommunications industry.
The Commissions proposed directive scales back the requirements somewhat, and would reimburse companies for their "demonstrated additional costs." Telcos would have to retain mobile and fixed telephony data, such as numbers called, for one year.
ISPs would be required to store logs of e-mail and Internet server traffic, including numbers dialed via IP telephony, for six months. Under the member states plans, the limit would be three years.
"This proposal is a very balanced and constructive one, which takes account of the fundamental rights to security, to a private life and protection of personal data, as well as different interests, in particular those of law enforcement authorities and communication providers," Franco Frattini, Commission vice president for Justice, Freedom and Security, said at a news conference.
He said the Commission expects an agreement with the European Parliament and member states by the end of the year. "We have no time to lose," he said.
Critics said the plan is mostly the same as the member states controversial proposal, with the exception of allowing for reimbursement and shorter retention periods. "Like the ... Council, the Commission fails to provide any evidence for the need and benefits of data retention," said civil liberties group EDRI (European Digital Rights) in a bulletin.
EDRI and more than 100 other human rights and civil liberties groups are backing a petition demanding the data retention plans be scrapped.
"No research has been conducted anywhere in Europe that supports the need and necessity of creating such a large-scale database containing such sensitive data for the purpose of fighting crime and terrorism," the petition says.
The Commissions proposed directive would be far less expensive for telecommunications companies than the vision put forward by member states. A report from LIBE, the parliamentary committee on Civil Liberties, Justice and Home Affairs, estimated that compliance with the member states proposal would cost each company 180 million euros a year.
The report concluded that the data acquired would be useless against crime and terrorism. "Given the volume of data to be retained, particularly Internet data, it is unlikely that an appropriate analysis of the data will be at all possible," the report said. Meanwhile, criminal rings and terrorists would have no problem finding ways to avoid being traced, according to the report.
The committee estimated that the member states plan would entail the collection of 20,000 to 40,000 terabytes of data per year, or the equivalent of 10 stacks of files each reaching from the Earth to the moon.
In addition, LIBE advised that the plan was probably illegal, as such requirements could only legally be put into place by the Commission and the European Parliament. The Councils legal advisers came to a similar conclusion.
Civil liberties organizers said the proposal placed an unnecessary burden on citizens and the telecoms industry. "The proposal should be withdrawn, as law enforcement agencies have all the powers they need," Tony Bunyan, editor of civil liberties group Statewatch, said in an analysis of the member states proposal.
So far, however, member states are continuing to press on with their initiative, which has one great advantage—it doesnt need the approval of the European Parliament, whose democratically elected members are susceptible to lobbying by telecommunications companies and civil rights groups.
On Sept. 27, the European Parliament made its views clear in a formal vote rejecting the states plan. In support of its decision, the body cited the danger to individual privacy and the financial burden to the industry. The vote is not binding and doesnt stop member states from forcing their plan through, if they can gain the backing of other EU national governments.
German liberal deputy Alexander Alvaro told Reuters that the Parliament may also look for modifications to the Commissions plan. Specifically, they may insist on a "sunset clause," under which the resulting legislation would expire after a certain period, unless reapproved by parliamentarians.
Editors Note: This story was updated to include information on the Sept. 27 vote.