Open source intelligence
Encyclopedia
Open-source intelligence (OSINT) is a form of intelligence collection management
that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence
. In the intelligence community
(IC), the term "open" refers to overt, publicly available sources (as opposed to covert or classified sources); it is not related to open-source software
or public intelligence.
OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.
(DoD), as "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement."
OSINT is, as of 2005, defined by the U.S. Office of Management and Budget under the category of "Forces And Direct Support" and specifically for the DoD under Commercial Code M320 as
A wide variety of vendors sell information products specifically within this category.
Open-source intelligence under one name or another has been around for hundreds of years. The significance today of OSINT in the USA is the conflict between military, government and the private sector as to how the bulk of intelligence should be obtained. With the Internet, instant communications, and advanced media search the bulk of actionable and predictive intelligence can be obtained from public, unclassified sources. Government agencies have been slow to embrace OSINT, or believe they already have suitable information feeds from the media, academia and public records.
OSINT is especially helpful in addressing global coverage, a term encompassing all of the countries and topics that are not considered by the secret or national security worlds to be "vital." Until recently, that included terrorism.
, focused and directed to specific industries still has opportunities for small and medium businesses to compete in niche markets, but that too is being consolidated by the major information providers. In the media OSINT is considered as nothing new, the everyday operation of a traditional newsroom. Investigative journalists use searches, databases, primary interviews, sources, and leaks to write every feature.
in most countries. Of course, espionage that is not treason (i.e. betraying one's country of citizenship) has been a tool of statecraft since ancient times, is widely engaged in by nearly all countries, and is considered an honorable trade. Most countries recognize this, and if their counterintelligence agencies capture a foreign spy, that spy is usually unceremoniously deported or traded back to their homeland (for other spies) after a hostile debriefing; actual execution or refusal to trade back foreign spies with non-official cover would result in consequences in bilateral relations of the gravest possible magnitude, being an extraordinarily hostile act, even if those consequences were unofficially and extrajudicially imposed.
(FBIS) was created in 1941 to access and exploit OSINT in relation to World War II. A classic example of their value and success is reflected in the price of oranges in Paris as an indicator of whether railroad bridges had been bombed successfully.
The recent history of OSINT began in 1988 when General Alfred M. Gray, Jr.
, Commandant of the Marine Corps, called for a redirection of US intelligence away from the collapsing Soviet Union and toward non-state actors and Third World zones of instability. Additionally, he pointed out that most of the intelligence which needs to be known could be obtained via OSINT, and recommended a substantive increase in resources for this aspect of the intelligence collection spectrum of sources.
In the fall of 1992, Senator David Boren, then Chairman of the Senate Select Committee on Intelligence, sponsored the National Security Act of 1992, attempting to achieve modest reform in the U.S. Intelligence Community. His counterpart on the House Permanent Select Committee on Intelligence
was Congressman Dave McCurdy
. The House version of the legislation included a separate open-source office, at the suggestion of Larry Prior, a Marine Reservist familiar with the MCIC
experience and then serving on the House Permanent Select Committee on Intelligence staff.
The Aspin-Brown Commission
stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.
In issuing its July 2004 report, the 9/11 Commission
recommended the creation of an open-source intelligence agency, but without further detail or comment. Subsequently, the WMD Commission
(also known as the Robb-Silberman Commission) report in March 2005 recommended the creation of an open-source directorate at the CIA.
Following these recommendations, in November 2005 the Director of National Intelligence
announced the creation of the DNI Open Source Center
. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery." In addition to collecting openly available information, it would train analysts to make better use of this information. The Center absorbed the CIA
's previously existing Foreign Broadcast Information Service
(FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the Center.
In December 2005, the Director of National Intelligence appointed Eliot A. Jardines
as the Assistant Deputy Director of National Intelligence for Open Source
to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the National Open Source Enterprise
. Mr. Jardines has established the National Open Source Enterprise and authored Intelligence Community Directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by Dan Butler
who is ADDNI/OS and previously Mr. Jardines' Senior Advisor for Policy.
The Library of Congress
sponsors the Federal Research Division
(FRD) which conducts a great deal of tailored open-source research on a fee-for-service basis for the executive branch.
) are dictated by Intelligence Community Directive 301 promulgated by the Director of National Intelligence. The Directive establishes the authorities and responsibilities of the Assistant Deputy Director of National Intelligence for Open Source (ADDNI/OS), the DNI's Open Source Center and the National Open Source Committee.
Prior to the establishment of the National Open Source Enterprise, the Foreign Broadcast Information Service
(FBIS), established in 1941, was the government's primary open-source unit, transcribing and translating foreign broadcasts. It absorbed the Defense Department's Joint Publications Research Service
(JPRS), which did a similar function with foreign printed materials, including newspapers, magazines, and technical journals.
encouraged in part by the Defense Science Board
reports on Strategic Communication
and Transition to and From Hostilities
, created the Defense Open Source Program (DOSP). The current Under-Secretary of Defense for Intelligence is assigned executive agency for this program to the Defense Intelligence Agency
.
U.S. military offices that engage in OSINT activities include:
Examples of successful law enforcement OSINT include Scotland Yard
OSINT; Royal Canadian Mounted Police
(RCMP) OSINT.
INTERPOL
and EUROPOL
experimented with OSINT units for a time, but they appear to have atrophied with the departure of their individual champions.
New York Police Department (NYPD) is known to have an OSINT unit, as does the Los Angeles County Sheriff's Department, housed within the Emergency Operations Bureau and affiliated with the LA Joint Regional Intelligence Center.
, Competitor Intelligence, and Business Intelligence
, and is often a chief area of practice of private intelligence agencies.
Businesses may use information broker
s and private investigator
s to collect and analyze relevant information for business purposes which may include the media
, deep web
, web 2.0
and commercial content.
Government Publications
Advocacy and analysis of OSINT
News and commentary
Information Security
Intelligence collection management
Intelligence Collection Management is the process of managing and organizing the collection of intelligence information from various sources. The collection department of an intelligence organization may attempt basic validation of that which it collects, but is not intended to analyze its...
that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence
Intelligence (information gathering)
Intelligence assessment is the development of forecasts of behaviour or recommended courses of action to the leadership of an organization, based on a wide range of available information sources both overt and covert. Assessments are developed in response to requirements declared by the leadership...
. In the intelligence community
Intelligence agency
An intelligence agency is a governmental agency that is devoted to information gathering for purposes of national security and defence. Means of information gathering may include espionage, communication interception, cryptanalysis, cooperation with other institutions, and evaluation of public...
(IC), the term "open" refers to overt, publicly available sources (as opposed to covert or classified sources); it is not related to open-source software
Open-source software
Open-source software is computer software that is available in source code form: the source code and certain other rights normally reserved for copyright holders are provided under a software license that permits users to study, change, improve and at times also to distribute the software.Open...
or public intelligence.
Open sources for intelligence
OSINT includes a wide variety of information and sources:- MediaMedia (communication)In communications, media are the storage and transmission channels or tools used to store and deliver information or data...
: newspapers, magazines, radio, television, and computer-based information. - Web-based communitiesWeb 2.0The term Web 2.0 is associated with web applications that facilitate participatory information sharing, interoperability, user-centered design, and collaboration on the World Wide Web...
and user-generated contentUser-generated contentUser generated content covers a range of media content available in a range of modern communications technologies. It entered mainstream usage during 2005 having arisen in web publishing and new media content production circles...
: social-networking sites, video sharing sitesVideo hosting serviceA video hosting service allows individuals to upload video clips to an Internet website. The video host will then store the video on its server, and show the individual different types of code to allow others to view this video...
, wikis, blogs, and folksonomies. - Public data: government reports, official data such as budgets, demographics, hearings, legislative debates, press conferences, speeches, marine and aeronautical safety warnings, environmental impact statements and contract awards.
- Observation and reporting: amateur airplane spotters, radio monitors and satellite observers among many others have provided significant information not otherwise available. The availability of worldwide satellite photography, often of high resolution, on the Web (e.g., Google Earth) has expanded open-source capabilities into areas formerly available only to major intelligence services.
- Professional and academic: conferences, symposia, professional associations, academic papers, and subject matter experts.
- Most information has geospatial dimensions, but many often overlook the geospatial side of OSINT: not all open-source data is unstructured text. Examples of geospatial open source include hard and softcopy maps, atlases, gazetteers, port plans, gravity data, aeronautical data, navigation data, geodetic data, human terrain data (cultural and economic), environmental data, commercial imagery, LIDARLIDARLIDAR is an optical remote sensing technology that can measure the distance to, or other properties of a target by illuminating the target with light, often using pulses from a laser...
, hyper and multi-spectral data, airborne imagery, geo-names, geo-features, urban terrain, vertical obstruction data, boundary marker data, geospatial mashupsMashup (web application hybrid)In Web development, a mashup is a Web page or application that uses and combines data, presentation or functionality from two or more sources to create new services...
, spatial databaseSpatial DatabaseA spatial database is a database that is optimized to store and query data that is related to objects in space, including points, lines and polygons. While typical databases can understand various numeric and character types of data, additional functionality needs to be added for databases to...
s, and web services. Most of the geospatial data mentioned above is integrated, analyzed, and syndicated using geospatial software like a Geographic Information System (GIS) not a browser per se.
OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.
Definers for OSINT
OSINT is defined by both the U.S. Director of National Intelligence and the U.S. Department of DefenseUnited States Department of Defense
The United States Department of Defense is the U.S...
(DoD), as "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement."
OSINT is, as of 2005, defined by the U.S. Office of Management and Budget under the category of "Forces And Direct Support" and specifically for the DoD under Commercial Code M320 as
- Open-source intelligence (OSINT) collection/processing
A wide variety of vendors sell information products specifically within this category.
Open-source intelligence under one name or another has been around for hundreds of years. The significance today of OSINT in the USA is the conflict between military, government and the private sector as to how the bulk of intelligence should be obtained. With the Internet, instant communications, and advanced media search the bulk of actionable and predictive intelligence can be obtained from public, unclassified sources. Government agencies have been slow to embrace OSINT, or believe they already have suitable information feeds from the media, academia and public records.
OSINT is especially helpful in addressing global coverage, a term encompassing all of the countries and topics that are not considered by the secret or national security worlds to be "vital." Until recently, that included terrorism.
Competitive intelligence
In the private sector competitive intelligenceCompetitive intelligence
A broad definition of competitive intelligence is the action of defining, gathering, analyzing, and distributing intelligence about products, customers, competitors and any aspect of the environment needed to support executives and managers in making strategic decisions for an organization.Key...
, focused and directed to specific industries still has opportunities for small and medium businesses to compete in niche markets, but that too is being consolidated by the major information providers. In the media OSINT is considered as nothing new, the everyday operation of a traditional newsroom. Investigative journalists use searches, databases, primary interviews, sources, and leaks to write every feature.
Risks for practitioners
Accredited journalists have some protection in asking questions, and researching for recognized media outlets. Even so they can be imprisoned, even executed, for seeking out OSINT. Private individuals illegally collecting data for a foreign military or intelligence agency is considered espionageEspionage
Espionage or spying involves an individual obtaining information that is considered secret or confidential without the permission of the holder of the information. Espionage is inherently clandestine, lest the legitimate holder of the information change plans or take other countermeasures once it...
in most countries. Of course, espionage that is not treason (i.e. betraying one's country of citizenship) has been a tool of statecraft since ancient times, is widely engaged in by nearly all countries, and is considered an honorable trade. Most countries recognize this, and if their counterintelligence agencies capture a foreign spy, that spy is usually unceremoniously deported or traded back to their homeland (for other spies) after a hostile debriefing; actual execution or refusal to trade back foreign spies with non-official cover would result in consequences in bilateral relations of the gravest possible magnitude, being an extraordinarily hostile act, even if those consequences were unofficially and extrajudicially imposed.
Value
According to the Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction report submitted in March 2005, OSINT must be included in the all-source intelligence process for the following reasons (as stated in the report):- The ever-shifting nature of our intelligence needs compels the IC to quickly and easily understand a wide range of foreign countries and cultures. – … today’s threats are rapidly changing and geographically diffuse; it is a fact of life that an intelligence analyst may be forced to shift rapidly from one topic to the next. Increasingly, IC professionals need to quickly assimilate social, economic, and cultural information about a country—information often detailed in open sources.
- Open-source information provides a base for understanding classified materials. Despite large quantities of classified material produced by the IC, the amount of classified information produced on any one topic can be quite limited, and may be taken out of context if viewed only from a classified-source perspective. Perhaps the most important example today relates to terrorism, where open-source information can fill gaps and create links that allow analysts to better understand fragmented intelligence, rumored terrorist plans, possible means of attack, and potential targets.
- Open-source materials can protect sources and methods. Sometimes an intelligence judgment that is actually informed with sensitive, classified information can be defended on the basis of open-source reporting. This can prove useful when policy-makers need to explain policy decisions or communicate with foreign officials without compromising classified sources.
- Only open source can store history. A robust open-source program can, in effect, gather data to monitor the world’s cultures and how they change with time. This is difficult, if not impossible, using the snapshots provided by classified collection methods.
Process
Information collection in OSINT is generally a different problem from collection in other intelligence disciplines where obtaining the raw information to be analyzed may be the major difficulty, particularly if it is to be obtained from non-cooperative targets. In OSINT, the chief difficulty is in identifying relevant, reliable sources from the vast amount of publicly available information. However, this is not as great a challenge for those who know how to access local knowledge and how to leverage human experts who can create new tailored knowledge on the fly.History
The Foreign Broadcast Information ServiceForeign Broadcast Information Service
Foreign Broadcast Information Service was an open source intelligence component of the Central Intelligence Agency's Directorate of Science and Technology. It monitored, translated, and disseminated within the U.S. government openly available news and information from media sources outside the...
(FBIS) was created in 1941 to access and exploit OSINT in relation to World War II. A classic example of their value and success is reflected in the price of oranges in Paris as an indicator of whether railroad bridges had been bombed successfully.
The recent history of OSINT began in 1988 when General Alfred M. Gray, Jr.
Alfred M. Gray, Jr.
Alfred M. Gray, Jr. , is a retired United States Marine Corps general who served as the twenty-ninth Commandant of the Marine Corps from 1987-91. He retired from the Corps in 1991 after 41 years of service.-Personal:Alfred M. Gray, Jr...
, Commandant of the Marine Corps, called for a redirection of US intelligence away from the collapsing Soviet Union and toward non-state actors and Third World zones of instability. Additionally, he pointed out that most of the intelligence which needs to be known could be obtained via OSINT, and recommended a substantive increase in resources for this aspect of the intelligence collection spectrum of sources.
In the fall of 1992, Senator David Boren, then Chairman of the Senate Select Committee on Intelligence, sponsored the National Security Act of 1992, attempting to achieve modest reform in the U.S. Intelligence Community. His counterpart on the House Permanent Select Committee on Intelligence
United States House Permanent Select Committee on Intelligence
The United States House Permanent Select Committee on Intelligence is a committee of the United States House of Representatives, currently chaired by Mike Rogers. It is the primary committee in the U.S...
was Congressman Dave McCurdy
Dave McCurdy
David Keith McCurdy is a U.S. lawyer, politician, conservative Democrat, and a former Oklahoma Congressman from Oklahoma's 4th congressional district. He is president of the American Gas Association.-Early life and education:...
. The House version of the legislation included a separate open-source office, at the suggestion of Larry Prior, a Marine Reservist familiar with the MCIC
Marine Corps Intelligence Command
The Marine Corps Intelligence Command was created in 1988 by then Commandant of the Marine Corps Alfred M. Gray, Jr.He created this organization to address the lack of expeditionary intelligence support for policy, acquisition, and operations, as each of the service intelligence centers focused...
experience and then serving on the House Permanent Select Committee on Intelligence staff.
The Aspin-Brown Commission
Aspin-Brown Commission
The Aspin-Brown Commission, more properly known as the Commission on the Roles and Capabilities of the US Intelligence Community, was commissioned by the United States Congress after the National Security Act of 1992 failed to be passed. The Commission produced a report in 1996. In the year 2000,...
stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.
In issuing its July 2004 report, the 9/11 Commission
9/11 Commission
The National Commission on Terrorist Attacks Upon the United States, also known as the 9/11 Commission, was set up on November 27, 2002, "to prepare a full and complete account of the circumstances surrounding the September 11, 2001 attacks", including preparedness for and the immediate response to...
recommended the creation of an open-source intelligence agency, but without further detail or comment. Subsequently, the WMD Commission
Iraq Intelligence Commission
The Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction is a panel created by Executive Order 13328 signed by U.S. President George W. Bush in February 2004...
(also known as the Robb-Silberman Commission) report in March 2005 recommended the creation of an open-source directorate at the CIA.
Following these recommendations, in November 2005 the Director of National Intelligence
United States Director of National Intelligence
The Director of National Intelligence , is the United States government official subject to the authority, direction and control of the President, who is responsible under the Intelligence Reform and Terrorism Prevention Act of 2004 for:...
announced the creation of the DNI Open Source Center
Open Source Center
The Director of National Intelligence Open Source Center is a United States government intelligence center that provides analysis of open source materials, including gray literature, through OSC's headquarters and overseas bureaus...
. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery." In addition to collecting openly available information, it would train analysts to make better use of this information. The Center absorbed the CIA
Central Intelligence Agency
The Central Intelligence Agency is a civilian intelligence agency of the United States government. It is an executive agency and reports directly to the Director of National Intelligence, responsible for providing national security intelligence assessment to senior United States policymakers...
's previously existing Foreign Broadcast Information Service
Foreign Broadcast Information Service
Foreign Broadcast Information Service was an open source intelligence component of the Central Intelligence Agency's Directorate of Science and Technology. It monitored, translated, and disseminated within the U.S. government openly available news and information from media sources outside the...
(FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the Center.
In December 2005, the Director of National Intelligence appointed Eliot A. Jardines
Eliot A. Jardines
Eliot A. Jardines was the first Assistant Deputy Director of National Intelligence for Open Source in the United States.-Early life and education:...
as the Assistant Deputy Director of National Intelligence for Open Source
Assistant Deputy Director of National Intelligence for Open Source
The Assistant Deputy Director of National Intelligence for Open Source is a senior level position within the Office of the Director of National Intelligence, responsible for developing strategic direction, establishing policy and managing fiscal resources for Open Source Intelligence, providing...
to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the National Open Source Enterprise
National Open Source Enterprise
The National Open Source Enterprise is a United States Government organization that was established by Assistant Deputy Director of National Intelligence for Open Source , Eliot A...
. Mr. Jardines has established the National Open Source Enterprise and authored Intelligence Community Directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by Dan Butler
Dan Butler (civil servant)
Dan Butler, born Daniel S. Butler, is the second Assistant Deputy Director of National Intelligence for Open Source . As such, he is responsible for developing strategic direction, establishing policy and managing fiscal resources for open source exploitation as well as document and media...
who is ADDNI/OS and previously Mr. Jardines' Senior Advisor for Policy.
Government
There are a large number of open-source activities taking place throughout the US Government. Frequently, these open-source activities are described as "media monitoring", "media analysis", "internet research" and "public surveys" but are open source nonetheless.The Library of Congress
Library of Congress
The Library of Congress is the research library of the United States Congress, de facto national library of the United States, and the oldest federal cultural institution in the United States. Located in three buildings in Washington, D.C., it is the largest library in the world by shelf space and...
sponsors the Federal Research Division
Federal Research Division
The Federal Research Division is the research and analysis unit of the United States Library of Congress.The Federal Research Division provides directed research and analysis on domestic and international subjects to agencies of the United States government, the District of Columbia, and...
(FRD) which conducts a great deal of tailored open-source research on a fee-for-service basis for the executive branch.
Intelligence
The US Intelligence Community's open-source activities (known as the National Open Source EnterpriseNational Open Source Enterprise
The National Open Source Enterprise is a United States Government organization that was established by Assistant Deputy Director of National Intelligence for Open Source , Eliot A...
) are dictated by Intelligence Community Directive 301 promulgated by the Director of National Intelligence. The Directive establishes the authorities and responsibilities of the Assistant Deputy Director of National Intelligence for Open Source (ADDNI/OS), the DNI's Open Source Center and the National Open Source Committee.
Prior to the establishment of the National Open Source Enterprise, the Foreign Broadcast Information Service
Foreign Broadcast Information Service
Foreign Broadcast Information Service was an open source intelligence component of the Central Intelligence Agency's Directorate of Science and Technology. It monitored, translated, and disseminated within the U.S. government openly available news and information from media sources outside the...
(FBIS), established in 1941, was the government's primary open-source unit, transcribing and translating foreign broadcasts. It absorbed the Defense Department's Joint Publications Research Service
Joint Publications Research Service
The Joint Publications Research Service was a United States government defense-funded organization that was absorbed into the Foreign Broadcast Information Service but its funding and personnel did not transfer...
(JPRS), which did a similar function with foreign printed materials, including newspapers, magazines, and technical journals.
Military
The former Under-Secretary of Defense for Intelligence, Dr. Stephen CamboneStephen Cambone
Stephen A. Cambone was the first United States Under-Secretary of Defense for Intelligence, a post created in March 2003. Cambone first came to the attention of the public at large during the testimony of Major General Antonio Taguba before the U.S...
encouraged in part by the Defense Science Board
Defense Science Board
The Defense Science Board is a committee of civilian experts appointed to advise the U.S. Department of Defense on scientific and technical matters...
reports on Strategic Communication
Strategic Communication
Strategic Communication can mean either communicating a concept, a process, or data that satisfies a long term strategic goal of an organization by allowing facilitation of advanced planning, or communicating over long distances usually using international telecommunications or dedicated global...
and Transition to and From Hostilities
Transition to and From Hostilities
The Defense Science Board report, Transition to and from Hostilities, published in December 2004, was the foundation for the new DoD Instruction 3000 that elevates Operations Other Than War to co-equal status with combat operations....
, created the Defense Open Source Program (DOSP). The current Under-Secretary of Defense for Intelligence is assigned executive agency for this program to the Defense Intelligence Agency
Defense Intelligence Agency
The Defense Intelligence Agency is a member of the Intelligence Community of the United States, and is the central producer and manager of military intelligence for the United States Department of Defense, employing over 16,500 U.S. military and civilian employees worldwide...
.
U.S. military offices that engage in OSINT activities include:
- Defense Intelligence AgencyDefense Intelligence AgencyThe Defense Intelligence Agency is a member of the Intelligence Community of the United States, and is the central producer and manager of military intelligence for the United States Department of Defense, employing over 16,500 U.S. military and civilian employees worldwide...
- National Geospatial-Intelligence AgencyNational Geospatial-Intelligence AgencyThe National Geospatial-Intelligence Agency is an agency of the federal government of the United States with the primary mission of collecting, analyzing and distributing geospatial intelligence in support of national security. NGA was formerly known as the National Imagery and Mapping Agency ...
- US Army Foreign Military Studies OfficeForeign Military Studies OfficeThe Foreign Military Studies Office, or FMSO, is a research and analysis center for the United States Army that is part of the United States Army Combined Arms Center at Fort Leavenworth. It manages the Joint Reserve Intelligence Center there.-Mission:...
- US Army Asian Studies Detachment
- EUCOM JAC Molesworth
- Open Source Branch, Joint Intelligence Center, U.S. Special Operations Command
- Foreign Media Monitoring in Support of Information Operations, U.S. Strategic Command
Homeland Security
The Department of Homeland Security has an active open-source intelligence unit. In congressional testimony before the House Homeland Security Committee's Intelligence, Information Sharing and Terrorism Risk Assessment Subcommittee the Undersecretary of Homeland Security Charles Allen indicated on February 14, 2007, that he had established the "Domestic Open Source Enterprise" to support the Department's OSINT needs and that of state, local and tribal partners.Law enforcement
The law enforcement OSINT community applies open-source intelligence (OSINT) to the prediction, prevention, investigation, and prosecution of criminals including terrorists.Examples of successful law enforcement OSINT include Scotland Yard
Scotland Yard
Scotland Yard is a metonym for the headquarters of the Metropolitan Police Service of London, UK. It derives from the location of the original Metropolitan Police headquarters at 4 Whitehall Place, which had a rear entrance on a street called Great Scotland Yard. The Scotland Yard entrance became...
OSINT; Royal Canadian Mounted Police
Royal Canadian Mounted Police
The Royal Canadian Mounted Police , literally ‘Royal Gendarmerie of Canada’; colloquially known as The Mounties, and internally as ‘The Force’) is the national police force of Canada, and one of the most recognized of its kind in the world. It is unique in the world as a national, federal,...
(RCMP) OSINT.
INTERPOL
Interpol
Interpol, whose full name is the International Criminal Police Organization – INTERPOL, is an organization facilitating international police cooperation...
and EUROPOL
Europol
Europol is the European Union's criminal intelligence agency. It became fully operational on 1 July 1999....
experimented with OSINT units for a time, but they appear to have atrophied with the departure of their individual champions.
New York Police Department (NYPD) is known to have an OSINT unit, as does the Los Angeles County Sheriff's Department, housed within the Emergency Operations Bureau and affiliated with the LA Joint Regional Intelligence Center.
Business
Business OSINT encompasses Commercial IntelligenceCommercial Intelligence
Commercial Intelligence is the highest and most comprehensive form of legal, ethical open source intelligence as practiced by diverse international and localized businesses....
, Competitor Intelligence, and Business Intelligence
Business intelligence
Business intelligence mainly refers to computer-based techniques used in identifying, extracting, and analyzing business data, such as sales revenue by products and/or departments, or by associated costs and incomes....
, and is often a chief area of practice of private intelligence agencies.
Businesses may use information broker
Information broker
An information broker, also known as an independent information professional or information consultant, is a person or business that researches information for clients...
s and private investigator
Private investigator
A private investigator , private detective or inquiry agent, is a person who can be hired by individuals or groups to undertake investigatory law services. Private detectives/investigators often work for attorneys in civil cases. Many work for insurance companies to investigate suspicious claims...
s to collect and analyze relevant information for business purposes which may include the media
Mass media
Mass media refers collectively to all media technologies which are intended to reach a large audience via mass communication. Broadcast media transmit their information electronically and comprise of television, film and radio, movies, CDs, DVDs and some other gadgets like cameras or video consoles...
, deep web
Deep web
The Deep Web refers to World Wide Web content that is not part of the Surface Web, which is indexed by standard search engines....
, web 2.0
Web 2.0
The term Web 2.0 is associated with web applications that facilitate participatory information sharing, interoperability, user-centered design, and collaboration on the World Wide Web...
and commercial content.
See also
- IntellipediaIntellipediaIntellipedia is an online system for collaborative data sharing used by the United States Intelligence Community . It was established as a pilot project in late 2005 and formally announced in April 2006 and consists of three wikis running on JWICS, SIPRNet, and Intelink-U...
- Open Source CenterOpen Source CenterThe Director of National Intelligence Open Source Center is a United States government intelligence center that provides analysis of open source materials, including gray literature, through OSC's headquarters and overseas bureaus...
- Private intelligence agency
- ROSIDSROSIDSROSIDS Rapid Open Source Intelligence Deployment System, which timeshifts the video then processes speech-to-text through the SAIL LABS Technology automatic speech recognition and then hands the XML result to a machine translation engine. The translated text is then resynchronized with the video...
- Special Libraries AssociationSpecial Libraries AssociationSpecial Libraries Association is an international professional association for library and information professionals working in business, government, law, finance, non-profit, and academic organizations and institutions....
- Strategic intelligenceStrategic intelligence1. Strategic intelligence pertains to the collection, processing, analysis, and dissemination of intelligence that is required for forming policy and military plans at the national and international level....
- NATO Open Source Intelligence HandbookNATO Open Source Intelligence HandbookThe NATO Open Source Intelligence Handbook is the standard reference available to the public. The other two NATO references are the NATO Open Source Intelligence Reader and the NATO Intelligence Exploitation of the Internet.-Sources:*...
, NATO Open Source Intelligence ReaderNATO Open Source Intelligence ReaderThe NATO Open Source Intelligence Reader is one of three standard references on Open source intelligence. The other two are the NATO Open Source Intelligence Handbook and the NATO Intelligence Exploitation of the Internet.-Sources:*... - PREDICT Open Source Intelligence TeamPREDICT Open Source Intelligence TeamPREDICT Open-Source Intelligence Team or POSIT is an open-source intelligence team at New Mexico State University. POSIT is part of NMSU's Center for Animal Health, Food Safety, and Bio-Security of the Agricultural Experiment Station at the College of Agricultural, Consumer and Environmental...
- ViTAPViTAPViTAP is a computer system that automatically can process video and audio news broadcasts in multiple languages. . It can transcribe news broadcasts into text. It is used in the MiTAP system, which is used by many organizations including the FBI....
- MiTAPMiTAPMiTAP, or Mitre Text and Audio Processing, is a computer system that tries to automatically gather, translate, organize, and present information "for monitoring infectious disease outbreaks and other global events." It is also used in the FBI Investigative Data Warehouse.Sources"Multiple...
- DARPA TIDES programDARPA TIDES programTIDES is an ambitious technology development effort, funded by DARPA. It stands for Translingual Information Detection, Extraction and Summarization. It is focused on the automated processing and understanding of a variety of human language data...
- Investigative Data WarehouseInvestigative Data WarehouseThe Investigative Data Warehouse, or IDW, is a searchable database operated by the FBI. It was created in 2004. Much of the nature and scope of the database is classified. The database is a centralization of multiple federal and state databases, including criminal records from various law...
- Fusion CenterFusion centerA fusion center is a terrorism prevention and response center, many of which were created under a joint project between the Department of Homeland Security and the US Department of Justice's Office of Justice Programs between 2003 and 2007....
- National Intelligence Open Source Committee
Literature
Scientific Publications- Arthuer S. Hulnick: ‘The Dilemma of Open Source Intelligence: Is OSINT Really Intelligence?’, pages 229–241, The Oxford Handbook of National Security Intelligence, 2010
- Cody Burke: ‘Freeing knowledge, telling secrets: Open source intelligence and development’, Bond University, May 2007
- Florian Schaurer, Jan Störger: ‘The Evolution of Open Source Intelligence’, OSINT Report 3/2010, ISN, ETH Zürich, October 2010
Government Publications
External links
General- The Open Source Intelligence Resource Discovery Toolkit
- The New Craft of Intelligence: Making the Most of Open Private Sector Knowledge
- Actual Intelligence Case Studies Leveraging Open Source Intelligence (OSINT)
- Sailing the Sea of OSINT in the Information Age
- The Intelligence Network
- OSINT discussion group at Yahoo!Yahoo!Yahoo! Inc. is an American multinational internet corporation headquartered in Sunnyvale, California, United States. The company is perhaps best known for its web portal, search engine , Yahoo! Directory, Yahoo! Mail, Yahoo! News, Yahoo! Groups, Yahoo! Answers, advertising, online mapping ,...
- Open Source Center – U.S. government arm focusing on open source intelligence under the DNI
- The OSINT Catalogue of reports and books
- Collection and Use of Open-Source Intelligence – A to Z
- Open Source Intelligence (OSINT): Issues for Congress, Congressional Research Service, December 5, 2007
- SAIL LABS Technology AG, Vienna - Austria
Advocacy and analysis of OSINT
- FindArticles.com, FMSO-JRIC and Open Source Intelligence: speaking prose in a world of verse, Military Intelligence Professional Bulletin, Oct–Dec, 2005 by Jacob W. Kipp
News and commentary
- WashTimes.com, Washington Times – CIA mines 'rich' content from blogs, 19 April 2006
- GCN.com, Government Computer News – Intelligence units mine the benefits of public sources 20 March 2006
- AFCEA.org, Article_Template.asp?articleid=1102&zoneid=31 Signal Magazine – Intelligence Center Mines Open Sources March 2006
- FindAcricles.com, Military Intelligence Professional Bulletin October–December, 2005 by Barbara G. Fast
- FAS.org, Congressional Testimony on OSINT and Homeland Security 21 June 2005
- FirstMonday.org, Open Source Intelligence by Stalder and Hirsh, 15 May 2002
- Forbes.com, When Everyone Can Mine Your Data by Taylor Buley, 11.21.08]
- http://www.nytimes.com/2006/12/03/magazine/03intelligence.html?pagewanted=print, Open-Source Spying, article from the New York Times, about open sources and wikis
- Cnet.com, Maltego and the science of 'open-source' snooping by Matt Asay, November 25, 2008
Information Security