Document Actions
Proposals
Up one level2006/03/13
Document Actions
Austin's Open Government Online Charter Amendment
A theoretical technical plan for satisfying the upcoming City charter amendment
There has been a petition filed to place on the May ballot an amendment to the City's charter that would force the city to put some of it's day-to-day business online. This charter amendment aims to bring accountability and transparency to the higher levels of municipal government, where the primary targets are City Council members, the City Manager, Assistant City Managers, division level managers and their respective staffs. The amendment requires some very specific conditions (I'll highlight the technical ones below) but first a disclaimer.
Although I currently work for the City of Austin, I am in no way representing or advocating *any* official position of the City in this matter. I am writing this as a concerned citizen of Austin who would like to see our government as transparent and efficient as possible and I am using information that is publicly available elsewhere to make my points.
With that in mind, this is a theoretical plan so assumptions have been made which could be incorrect. At this time, I am neither for nor against the proposed Charter amendment. My focus with this post is to try and bring a technical solution to the mandated goals of the Charter amendment in which to serve as a base for discussion with the Austin community. This opinion is put forward in the hopes it will facilitate discussion and will bring other concerned citizens into the mix and to showcase the alternative technical ways to satisfy the Charter amendment if passed. So please, do not read anything into this post that is not explicitly stated.
And with that, off we go. Here are the relevant sections (italics are direct quotes from the amendment; regular text is my opinion):
- The City must, as expeditiously as possible and to the greatest extent practical, make all public information available online in real time and accessible to the public.
- OPEN ACCESS TO CITY BUSINESS
- Within one year of the date this Amendment takes effect, applications and proposals for any permit or contract of significant value must be provided to the City in an electronic format.
- The City must maintain a system for electronic notification (such as email lists) to interested persons of any event or new information relating to the matter.
- All written communications between the City and the applicant relating to the matter must be posted online in real time in a manner searchable by the public.
- OPEN ACCESS TO CITY CALENDARS
- For all matters involving City business, the following people must maintain calendars of all meetings and maintain logs of all telephone calls: (a) City Councilmembers and their staff; (b) City Manager and his or her staff; (c) Assistant City Managers and their staff; and (d) all department heads.
- Calendars and logs must be posted online in real time and be accessible to the public.
- This provision must be implemented within six months of approval of this amendment.
- OPEN ACCESS TO CITY ELECTRONIC COMMUNICATIONS
- In order to better preserve written electronic communication for public disclosure, the City must establish a system that automatically archives all incoming and outgoing electronic communication that deals with City business to and from the following people in their official capacity: (a) City Councilmembers and their staff; (b) City Manager and his or her staff; (c) Assistant City Managers and their staff; and (d) all department heads.
- The above people are prohibited from discussing City business via any form of written electronic communication, such as a private email account, that is outside of the City’s automatic archiving system.
- OPEN ACCESS TO CITY FUNCTIONS
- All public information that has previously been released to someone making a public information request and which, because of the nature of the subject matter, the City determines is or is likely to become the subject of a subsequent public information request for substantially the same information
- EFFECTIVE ACCESS TO INFORMATION
- The City must create and maintain online tables of contents and indexes to enable the general public to easily find and access online City documents and public information. Information must be searchable, and be able to be located by author/submitter, individual recipient, date, and subject matter.
- Jabber - This will be the core transport piece and the legacy application data connector. Jabber is essentially an XML router and can act as the "bridge" from older client/server applications to the charter mandated goals.
- Zimbra - This is the email/calendaring piece. At its heart, it's an XML processing engine.
- Zope/Plone - This provides the public presentation framework as well as the internal data workflow paths for data classification and dissemination.
- OpenOffice.org - This would be the city's standard productivity application for creating/editing data. All data would be stored in OpenDocument format which is basically a zipped XML file (well, actually, it's 3 files zipped into one). All citizen correspondence with the city would also be in OpenDocument format.
- OPEN ACCESS TO CITY BUSINESS
- Within one year of the date this Amendment takes effect, applications and proposals for any permit or contract of significant value must be provided to the City in an electronic format.
- The City must maintain a system for electronic notification (such as email lists) to interested persons of any event or new information relating to the matter.
- All written communications between the City and the applicant relating to the matter must be posted online in real time in a manner searchable by the public.
- OPEN ACCESS TO CITY CALENDARS
- For all matters involving City business, the following people must maintain calendars of all meetings and maintain logs of all telephone calls: (a) City Councilmembers and their staff; (b) City Manager and his or her staff; (c) Assistant City Managers and their staff; and (d) all department heads.
- Calendars and logs must be posted online in real time and be accessible to the public.
- This provision must be implemented within six months of approval of this amendment.
- OPEN ACCESS TO CITY ELECTRONIC COMMUNICATIONS
- In order to better preserve written electronic communication for public disclosure, the City must establish a system that automatically archives all incoming and outgoing electronic communication that deals with City business to and from the following people in their official capacity: (a) City Councilmembers and their staff; (b) City Manager and his or her staff; (c) Assistant City Managers and their staff; and (d) all department heads.
- The above people are prohibited from discussing City business via any form of written electronic communication, such as a private email account, that is outside of the City’s automatic archiving system.
- OPEN ACCESS TO CITY FUNCTIONS
- All public information that has previously been released to someone making a public information request and which, because of the nature of the subject matter, the City determines is or is likely to become the subject of a subsequent public information request for substantially the same information
- EFFECTIVE ACCESS TO INFORMATION
- The City must create and maintain online tables of contents and indexes to enable the general public to easily find and access online City documents and public information. Information must be searchable, and be able to be located by author/submitter, individual recipient, date, and subject matter.
- Cost - There would be a hefty upfront cost for hardware and existing data conversion. The current official estimate is $24 million one-time cost and $11 million ongoing. Without getting into the specifics of cost, I feel the official costs might be slightly high as I think there could be some savings in one-time consulting fees, software licensing costs and ongoing staffing (possibly as much as $6mil in the one-time category). But the costs to implement a system like this would be high, make no mistake about it. This nut can't be cracked cheaply. One thing that I'd like to bring up which isn't specifically called for in the Charter amendment is data conversion costs to the OpenDocument format. Although the Charter amendment does not call for all city data to be accessible online, I would like to mention that it would be time and money well-spent. Although I couldn't find an "official" conversion efficiency rate for Microsoft Office to OpenDocument, I have found numerous articles (here's one) hinting the conversion rates are quite high. My own testing of the few hundred Microsoft formatted documents on my hard drive showed about a 90% conversion rate (by that I mean 9 out of 10 did not complain about conversion and the docs I opened up looked formatted correctly). So working on the assumption there is an effective 85% conversion rate, if the city has 2.5 million documents, that leaves 375,000 documents that would need some form of "hand-holding" to convert. If one employee can convert 20 documents a day, they would get through the problem documents in 51.37 years. To bring this down into the timeframe required by the Charter amendment (just for argument's sake: 1 year), the city would contract close to 50 people for document conversion (50 x $35,000/yr for a temp = $1.75 million). To me, this is a relative bargain as the city would have the bulk of it's data in a non-proprietary, easily convertible and manipulative format. Now, there are much better people than I to figure out total costs for a plan like this (and I would love to work with someone on this, just email me) but suffice to say, the city's official $24mil estimate would be sufficient to cover the costs of this plan. The city's official cost estimate provides for new positions (which would be needed for more python programmers, etc) so I think the cost estimate is a good attempt to "herd cats" but could be just a bit high. The silver lining is that the city should be able to recoup some (not all) of the one-time costs over the long-term with the savings of having the data in machine manipulative formats, an efficient and defined data workflow path (enhancing employee productivity) and the ongoing savings in license fees for the proprietary software replacements.
- Momentum - By that I mean that getting any large organization to change takes time. There are sure to be groups/departments/managers that are hostile to these types of ideas. But the Charter amendment demands specific goals within specific timeframes so this is the best type of pressure for a government agency (external pressure is always the most efficient). Having worked in government for the bulk of my career, I can make a safe assumption that a plan this agressive will meet with much resistance internally. It may very well take upwards of the first proposed year just to convince the parties targetted by this amendment to agree to the changes of workflow. But with public pressure and hopefully, media coverage, a change like this can be successful.
Revision history:
- 3/15/06 - Changed FOIA requests to Open Records requests from a reader's comment.
- Category(s)
- Government
- Open Source
- Proposals
2006/03/27
More technical discussion of Austin's Open Government Online Initiative
My last post put together a theoretical system that should be able to handle a near "real-time" posting of information in an online format if required. It was meant merely as a basis for discussion; to set the technical parameters of a system so we can break down specifically and in more detail, the functionality of the system in specific situations. In this post, I'll try to highlight and explain exactly how this system *should* work in situations that have been mentioned publicly. But first, a standard disclaimer:
Although I currently work for the City of Austin, I am in no way representing or advocating *any* official position of the City in this matter. I am writing this as a concerned citizen of Austin who would like to see our government as transparent and efficient as possible and I am using information that is publicly available elsewhere to make my points.
With that in mind, this is a theoretical plan so assumptions have been made which could be incorrect. At this time, I am neither for nor against the proposed Charter amendment. My focus with this post is to try and bring a technical solution to the mandated goals of the Charter amendment in which to serve as a base for discussion with the Austin technical community. This opinion is put forward in the hopes it will facilitate discussion and will bring other concerned citizens into the mix and to showcase the alternative technical ways to satisfy the Charter amendment if passed. So please, do not read anything into this post that is not explicitly stated.
I'll first discuss the situations mentioned in the ballot language proposed by the City Council. Here is the language that will be going on the ballot this May (italics are direct quotes; my responses are normal text. Further, I've changed the format of the language to make the points more clear [I don't think anyone can read big, run-on sentences very well]):
Please keep in mind that the service contracts will cost money as well as hardware would need to be purchased (the city's estimate on server hardware is fair and reflective; the city would not have to replace 1,540 PCs under this model saving an additional $1.2mil). Without delving further into specifics, between just EDIMS licensing and the replacement PC costs, there is a savings of $7.1mil off the projected one-time cost of $24mil and at least $1.1mil off the recurring cost of $11.6mil. (I feel the ballot language is wrong when it states $36mil upfront costs. That figure combines the one-time cost at $24mil and the recurring cost of $11.6mil. The recurring cost would not have an effect on the one-time purchasing cost of the first year. Again, if anyone is interested in fully fleshing the costs of this model out, please email me.)
So, in sum, I hope that the technical model I propose shows the flexibility and expandability sought by the OGO amendment to provide a transparent window into the people's government. I would like to re-emphasize that this model should *not* be taken as a roadmap but merely as a discussion of the salient technical issues the amendment seeks.
City LanguageFirst, let me say that there are those that feel the ballot language is not an accurate portrayal of the amendment's goals. I won't comment publicly my thoughts on the language at this time but I will address the technical concerns of the ballot language in the proposed system.
Shall the city charter be amended:
- to require that all private citizens emails to any public official be placed on the city website in real time, including emails or electronic communications between private citizens and public officials in all City departments, including the library department, police department, city health clinics and city departments handling utility bills and code enforcement, and limit the ability of citizens to keep private the details of these communications;
- to require that the heads of all city departments, including the police department, parks department, library department, all city manager s staff and all city council members and their staff post online in real time information about all meetings and phone calls with private citizens;
- prohibit the city from exercising state law protection for information that could expose the city and taxpayers to greater financial and legal liability and risk;
- to require the city to create at taxpayer expense an online electronic data system for most city communications and documents, which for the most part are already available to the public; and
- to install and permanently operate such a system at an estimated cost of approximately $36 million initially and $12 million annually thereafter if fully implemented, which could require a tax increase equivalent to three cents per $100 valuation or a reduction in city services?
a) to require that all private citizens emails to any public official be placed on the city website in real time, including emails or electronic communications between private citizens and public officials in all City departments, including the library department, police department, city health clinics and city departments handling utility bills and code enforcement, and limit the ability of citizens to keep private the details of these communications;Email is already subject to Open Records requests and state document retention policies so the only difference in this clause from a restatement of current policy is the "real time" bit (which I will address). Further, the amendment does not require breaking any existing federal or state laws in its effort for transparency (i.e- City health clinics, police departments and utilities are bound by federal laws like HIPAA and the USA Patriot act which supercede the City Charter's authority for online posting of information.) Privacy laws (whether federal or state) still supercede any authority the city charter may have (an important concept that seems lost within the context of the current ballot language). The relevant section(s) from the amendment is:
SECTION 2: Privacy Protected. Nothing within this amendment should be interpreted in a manner that would violate an individual’s existing constitutional or common law rights to privacy.and
(C)OPEN ACCESS TO CITY ELECTRONIC COMMUNICATIONSThe amendment merely calls for an archiving system for the above personnel and does not require "real time" capabilities. The city currently follows all state document retention laws which require email archiving for a period of no less than 2 years. As it is correct to legally interpret the language of the amendment as broadly as possible, the current ballot language does not recognize that the ultimate authority of "real time" resides with the city council and not with this amendment. Interpreting the amendment as broadly as possible, it is correct to assume that "real time" posting of email would be a goal if that action was practical (that is the keyword from the amendment). The city council is the determining body of the "practicalness" of any issue that falls outside the explicit conditions set by the amendment. Hence, in this limited example of "real time" email, because it falls outside the explicit condition set by the amendment, it should be referred to council for determination of how practical the idea is (the key clause from the amendment is "expeditiously as possible and to the greatest extent practical,"). If the council found it was practical, it would then be up to the council to determine the action to best fit the situation. Further, the OGO amendment only mentions "real time" in two explicit circumstances; calendars of the city council, city managers, division heads and their staff and written communications between the City and businesses and individuals seeking ecomonic development benefits. Anything outside the scope of the following sections would be referred to council for a practical definition. The amendment *does not* require "real time," online posting of email (even though the press continually miss this point).
(1)In order to better preserve written electronic communication for public disclosure, the City must establish a system that automatically archives all incoming and outgoing electronic communication that deals with City business to and from the following people in their official capacity:
- City Councilmembers and their staff;
- City Manager and his or her staff;
- Assistant City Managers and their staff; and
- all department heads.
SECTION 3: Open Government Online(I like to think of this situation as the city council's "Mars Mission." Even though a manned mission to Mars is possible, it still is not practical and the council could rule for various reasons [the cost is prohibitively expensive; no one is trained for the mission; the chemical rockets are not powerful enough to make it to Mars, the citizens don't want the extra $.03 per $100 valuation tax increase, etc] that the "Mars Mission" would not take place.) The proposed system could meet the goal of "real time" posting of email if required by council at a later date. On email reception, the email is "tagged" with an OGO compliant label (the label is contained within the metadata for that message). Retrieval and display of those messages would be a simple ad-hoc query from the Zope application server to the Zimbra MySQL-based metadata store for the appropriate tag (pseudo-SQL: SELECT msg WHERE msg.tag MATCHES "OGO" AND msg.user MATCHES "Council Member"). Dependent on the tags, messages could also be grouped so a query could pull up not just one member's email messages, but a whole group (psuedo-SQL: SELECT msgs WHERE msg.tag MATCHES "OGO" AND msg.group MATCHES "city council" WHERE msg.date MATCHES "last week"). The nice thing about this approach is that by tagging the messages with metadata as they come in, we can do efficient ad-hoc querying from the Zope server to the message store while utilizing a single data repository. Furthering enhancing system performance, Zope will cache any queries on the Zope server so multiple queries for the same data will not need multiple "trips" to the message store. I am having trouble finding where the council feels that the amendment is " ... limit(ing) the ability of citizens to keep private the details of these communications;". The closest section I can find is the waiver of rights in section 4(B). That section does not deal with the public-at-large though; it only deals with those individuals and businesses that are seeking economic benefits from the city and does not deal with private citizens email. I'm at a loss for where the council found that clause applicable under the "real time" section. The next section from the ballot language:(A)OPEN ACCESS TO CITY BUSINESS(4)All public information concerning the matter subject to Section 3(A)(2) must be posted to the website. All written communications between the City and the applicant relating to the matter must be posted online in real time in a manner searchable by the public.(B)OPEN ACCESS TO CITY CALENDARS(3)Calendars and logs must be posted online in real time and be accessible to the public.
b) to require that the heads of all city departments, including the police department, parks department, library department, all city manager s staff and all city council members and their staff post online in real time information about all meetings and phone calls with private citizens;And the relevant section(s) from the amendment:
(B)OPEN ACCESS TO CITY CALENDARSIn the proposed model, this requirement is already addressed in the current product line. Zimbra has partnered with another open source product, Asterisk to provide VoIP and PBX functionality within the collaboration server. Asterisk is a full-featured, enterprise-class PBX so the logging of calls in or out should be easy and display of those calls should be the same as the calendar view (more information is here and here.) The next section from the ballot language:
- For all matters involving City business, the following people must maintain calendars of all meetings and maintain logs of all telephone calls: (a) City Councilmembers and their staff; (b) City Manager and his or her staff; (c) Assistant City Managers and their staff; and (d) all department heads.
- These calendars and logs must contain the time, date, subject matter, and persons involved in all meetings and telephone calls involving City business. These calendars must be used to schedule and record all past and future meetings that occur after the implementation date of this section.
- Calendars and logs must be posted online in real time and be accessible to the public.
- “Meetings” includes all informal and formal meetings including but not limited to telephone conferences, videoconferences, happy hours, and luncheons.
- This provision must be implemented within six months of approval of this amendment.
c) prohibit the city from exercising state law protection for information that could expose the city and taxpayers to greater financial and legal liability and risk;There are no technical issues required within this clause but, for completeness, here are the relevant amendment sections:
SECTION 4: Public Information. The term “public information” means information that is required to be produced under Texas Government Code § 552.021. Public information also includes the following categories that must be produced in response to a public information request:I'll address the next two ballot language sections together:
- INFORMATION RELATING TO CIVIL LITIGATION. That the City is a party to litigation does not render information relating to that litigation less important; rather it often means the information is a matter of heightened public interest. Therefore, the City must not withhold information relating to civil litigation under Texas Government Code § 552.103, but it may withhold under other Public Information Act exceptions.
- ECONOMIC DEVELOPMENT INFORMATION. Information relating to economic development assistance or incentives is public information to which the public has a right of access.
- The City must require all businesses and individuals seeking to engage in the type of economic development negotiations referenced in Texas Government Code § 552.131 to execute and deliver to the City a waiver of any rights to prevent the public disclosure of all information exchanged with the City. The City is without authority to engage in economic development negotiations with any company that has not first executed a waiver.
- The City is without authority to shield economic development offers under Texas Government Code § 552.131(b).
- Nothing in section 4(B) prevents a City from withholding documents under Texas Government Code §§ 552.104, 552.105, or 552.108.
- AGENCY MEMORANDA. Open government in Austin ensures the people have access not only to the final decisions made by government officials but also to the process by which those decisions are made. The City must not use Texas Government Code § 552.111 to withhold information reflecting advice, opinion, and recommendations on policymaking matters, except the City may withhold attorney work product.
- PERSONNEL FILES. The City of Austin must not maintain an optional personnel file as authorized under Texas Local Government Code § 143.089(g) for employees of the Austin Police Department, nor does the City have authority to enter into any meet and confer or other agreement with any police officer association that requires creation or maintenance of a separate file that is closed to the public.
- EMAILS RELATED TO CITY BUSINESS. Email or other written electronic communication to or from a public official concerning City business is public information, including communications to or from privately owned email accounts or computers.
The proposed model does not require any new systems to be purchased outside of the one's listed in my first post. Jabber serves as a "bridge" to connect the various existing datasources into the display architecture while the "agents" speak the "native" application language to the existing datastores (once again, I'll mention the CAPWin project of Washington, DC.) Some of the cost in the City's official estimate is for software licensing ($6mil alone just for the document management system). As I've stated in my past post, the cost estimate from the city is a good attempt at "herding cats" but until specifics are laid out, noone actually knows how much the initiative will cost. For example, the model I propose is heavier into programming than software licensing. Most of the products I've used in this model do not have an initial or ongoing software licensing cost (however, service contracts will still need to be procured). So it can be assumed that this model will be less expensive in software licensing than the city's official estimate. I have yet to fully breakdown the initial and ongoing costs for this model (I'm more concerned about the technical issues and not cost; I leave that for people more skilled than I at cost estimation) but a quick rundown of the products listed and their software costs:
- to require the city to create at taxpayer expense an online electronic data system for most city communications and documents, which for the most part are already available to the public; and
- to install and permanently operate such a system at an estimated cost of approximately $36 million initially and $12 million annually thereafter if fully implemented, which could require a tax increase equivalent to three cents per $100 valuation or a reduction in city services?
| Cost estimation | |||
|---|---|---|---|
| Product | Initial Software Cost | Ongoing Licensing Cost | Service Contract required |
| Zope | $0 | $0 | Yes |
| Plone | $0 | $0 | Yes |
| Jabber | $0 | $0 | Yes |
| Jabber agents | $Unknown. Based on how many systems need to be tied in. | $0 | Yes |
| Zimbra | $28/user | $0 | Yes |
| OpenOffice.org | $0 | $0 | Yes |
- Category(s)
- Government
- Open Source
- Proposals
- Local Government
- Austin
- Open Government


Sounds like a great proposal! Best of luck! Jonathan Schwartz's blog, at sun.com, recently talked about the unfortunate events (to say the least) that can happen when government doesn't use open source or freely available applications.
http://blogs.sun.com/roller/page/jonathan?entry=why_odf_matters
Transitioning users to new software can be interesting, so of course be sure everyone knows the benefits, gets shown the software and trained early and often, etc. These are my thoughts and recommendations based on my experience.
http://openoffice.blogs.com/openoffice/2005/10/transitioning_a.html
Best of luck!
This is interesting, Scott, and a great, cool suggestion to think of using open source software, but I'm not sure I understand - the City's official estimate is $36 million, not $24 million. Are we looking at the same one?
Another problem, as with the City's estimate, is that you're assuming the amendment applies much more broadly than it really does, which is a political and legal, not a technical question. There are several categories of information that are REQUIRED to be put online (or in the case of email, to be archived), while the preamble says other public information must be put online only when the city council decides, through implementing ordinances THEY'LL write, that it is "possible," "practical" and not violative of anyone's privacy rights to do so. That won't be every scrap of paper. So don't assume "all" documents must go online, assume the documents REQUIRED to go online will, and that others will only when it possible, practical, etc.
Thus a proposal for implementing the amendment should estimate a minimalist version that only would put online only what's required, as well as estimate for a full-blown cadillac version that you and Peter Collins describe, but which the amendment does not mandate. Some categories of information were already going online, the AMANDA system for example, they just weren't going to give the public a password.
Also, the amendment would only apply going forward and would require no historic information to be converted - there's no need to apply any "converstion" rate to the total number of existing city documents, only new, incoming documents that aren't already transmitted electronically. Finally, the one-year time frame only applies to a few specific items, not "all" public information, so be sure to estimate costs only for those specific items on the front end as mandated, not would-be-nice stuff like universal conversion. An open-source proposal for implementing the amendment under those more minimalist assumptions would be a great mitzvah to the community.
See these items for more on what those more modest assumptions would look like:
http://opengovaustin.blogspot.com/2006/03/on-privacy-and-practicality-must.html
and
http://www.aclutx.org/article.php?aid=201
and
http://www.cleanwater-cleangovernment.org/debunking.cfm
Thanks for thinking so hard on this and I'd be interested if, after looking more closely at the specific requirements, changing those assumptions would significantly change your thinking about the total cost estimate. Regards,