Real-time Federal Post-Employment Notices Methodology
This document describes the process we use to scrape and ingest information from post-employment notifications released by the Senate Office of Public Record and the House Clerk's Office.
Background
The Honest Leadership and Open Government Act of 2007 requires the House clerk and the secretary of the Senate to notify departing members and staffers of any applicable post-employment restrictions and to post on their websites a list of these notifications.
This feed combines those Senate and House lists and makes it easier to see individuals whose restriction periods are ending soon. In addition, an RSS feed is available to keep track of whose restrictions will be expiring
Live updates
The data available through this tracker is updated one a week. The original sources are updated once every pay period (about two weeks).
De-duplication and Entity Resolution
Each time this data is updated, the system performs an automated de-duplication task, which resolves entities (former employees and the offices they were employed with) whose names appear slightly differently across many filings. This allows users to browse by any of the entities mentioned in the filing, to get a better sense of the history of that individual or organization's history of filing.
New Identifiers
The identifiers assigned to people and organizations in this database are a break from the wider Influence Explorer identifier scheme, and represent our first move toward using Open Civic Data identifiers. For now, they only uniquely identify entities found in post-employment notices, but will, in the future, allow users to view related records across all of Influence Explorer's datasets.
What Could Go Wrong
Because the entity resolution process is automated, it's possible that it might match two individuals or organizations that ought to be treated as separate entities. We're refining this process to limit our error rate, but if you see something tha tshould be fixed, please let us know! Use the feedback widget on the right side of the page, and please send us as much information as you have about the data you'd like to see fixed.
