Adversarially (non-)robust machine learning

* Register (or log in) to the AI4G Neural Network to add this session to your agenda or watch the replay

One of the key limitations of deep-learning is its inability to generalize to new domains. The focus of this talk will be on adversarial examples; inputs constructed by an adversary to mislead a machine-learning model. These adversarial examples can, for example, cause self-driving cars to misrecognize street signs or misidentify pedestrians.

This talk introduces how adversarial examples are generated and why they are so easy to find. Then, we consider recent attempts at increasing the robustness of neural networks. Across recent papers, we have studied several dozen defences proposed at top machine-learning and security conferences and found that almost all can be evaded and offer nearly no improvement on top of the undefended baselines. Worryingly, our most recent breaks require no new attack ideas and merely re-use earlier attack approaches.

General robustness is still a challenge for deep-learning and one that will require extensive work to solve.

WHAT IS TRUSTWORTHY AI SERIES?

Artificial Intelligence (AI) systems have steadily grown in complexity, gaining predictivity often at the expense of interpretability, robustness and trustworthiness. Deep neural networks are a prime example of this development. While reaching “superhuman” performances in various complex tasks, these models are susceptible to errors when confronted with tiny (adversarial) variations of the input – variations which are either not noticeable or can be handled reliably by humans. This expert talk series will discuss these challenges of current AI technology and will present new research aiming at overcoming these limitations and developing AI systems which can be certified to be trustworthy and robust.

The expert talk series will cover the following topics:

Measuring Neural Network Robustness
Auditing AI Systems
Adversarial Attacks and Defences
Explainability & Trustworthiness
Poisoning Attacks on AI
Certified Robustness
Model and Data Uncertainty
AI Safety and Fairness

The Trustworthy AI series is moderated by Wojciech Samek, Head of AI Department at Fraunhofer HHI, one of the top 20 AI labs in the world.

Panelists
Resources

Presentation - Nicholas Carlini

Shownotes

0:05:00 Start of presentation: Adversarially (non-)robust machine learning

0:05:26 How powerful is AI and how useful is it?

0:06:40 What if you deploy a model in the real world and there is an adversary?

0:09:06 When do we need machine learning models to be robust?

0:11:45 How do we generate adversarial examples?

0:18:20 Let’s defend against adversarial examples

0:20:40 How are adversarial attacks today compared to 2018?

0:24:00 How are adversarial attacks reused?

0:26:40 The problem of adversarial attacks is methodological

0:33:30 What do adversarial loss functions look like?

0:36:50 What’s next in fighting adversarial attacks?

0:39:00 Can the status of fighting against adversarial attacks be compared with the status of cryptography in the 1990s?

0:44:57 Claim: we are crypto pre-Shannon

0:48:26 Brief conclusion

0:49:30 Question: Does the bottom-up approach of neural networks have something to do with the vulnerability?

0:54:28 Question: Is there a trade-off between robustness and accuracy?

0:57:48 Question: Can explainability help defend against adversarial attacks?

0:59:13 Question: What is one of the most promising approaches to fight against adversarial attacks?

1:02:00 Question: Can we get rid of some of the vulnerability of neural networks by moving towards more generative models?

1:06:22 Question: Can we optimise robustness with the probability that an attack can happen?

Country or Area	ISO-alpha2 Code	ISO-alpha3 Code	Developed / Developing regions
Algeria	DZ	DZA	Developing
Egypt	EG	EGY	Developing
Libya	LY	LBY	Developing
Morocco	MA	MAR	Developing
Sudan	SD	SDN	Developing
Tunisia	TN	TUN	Developing
Western Sahara	EH	ESH	Developing
British Indian Ocean Territory	IO	IOT	Developing
Burundi	BI	BDI	Developing
Comoros	KM	COM	Developing
Djibouti	DJ	DJI	Developing
Eritrea	ER	ERI	Developing
Ethiopia	ET	ETH	Developing
French Southern Territories	TF	ATF	Developing
Kenya	KE	KEN	Developing
Madagascar	MG	MDG	Developing
Malawi	MW	MWI	Developing
Mauritius	MU	MUS	Developing
Mayotte	YT	MYT	Developing
Mozambique	MZ	MOZ	Developing
Réunion	RE	REU	Developing
Rwanda	RW	RWA	Developing
Seychelles	SC	SYC	Developing
Somalia	SO	SOM	Developing
South Sudan	SS	SSD	Developing
Uganda	UG	UGA	Developing
United Republic of Tanzania	TZ	TZA	Developing
Zambia	ZM	ZMB	Developing
Zimbabwe	ZW	ZWE	Developing
Angola	AO	AGO	Developing
Cameroon	CM	CMR	Developing
Central African Republic	CF	CAF	Developing
Chad	TD	TCD	Developing
Congo	CG	COG	Developing
Democratic Republic of the Congo	CD	COD	Developing
Equatorial Guinea	GQ	GNQ	Developing
Gabon	GA	GAB	Developing
Sao Tome and Principe	ST	STP	Developing
Botswana	BW	BWA	Developing
Eswatini	SZ	SWZ	Developing
Lesotho	LS	LSO	Developing
Namibia	NA	NAM	Developing
South Africa	ZA	ZAF	Developing
Benin	BJ	BEN	Developing
Burkina Faso	BF	BFA	Developing
Cabo Verde	CV	CPV	Developing
Côte d’Ivoire	CI	CIV	Developing
Gambia	GM	GMB	Developing
Ghana	GH	GHA	Developing
Guinea	GN	GIN	Developing
Guinea-Bissau	GW	GNB	Developing
Liberia	LR	LBR	Developing
Mali	ML	MLI	Developing
Mauritania	MR	MRT	Developing
Niger	NE	NER	Developing
Nigeria	NG	NGA	Developing
Saint Helena	SH	SHN	Developing
Senegal	SN	SEN	Developing
Sierra Leone	SL	SLE	Developing
Togo	TG	TGO	Developing
Anguilla	AI	AIA	Developing
Antigua and Barbuda	AG	ATG	Developing
Aruba	AW	ABW	Developing
Bahamas	BS	BHS	Developing
Barbados	BB	BRB	Developing
Bonaire, Sint Eustatius and Saba	BQ	BES	Developing
British Virgin Islands	VG	VGB	Developing
Cayman Islands	KY	CYM	Developing
Cuba	CU	CUB	Developing
Curaçao	CW	CUW	Developing
Dominica	DM	DMA	Developing
Dominican Republic	DO	DOM	Developing
Grenada	GD	GRD	Developing
Guadeloupe	GP	GLP	Developing
Haiti	HT	HTI	Developing
Jamaica	JM	JAM	Developing
Martinique	MQ	MTQ	Developing
Montserrat	MS	MSR	Developing
Puerto Rico	PR	PRI	Developing
Saint Barthélemy	BL	BLM	Developing
Saint Kitts and Nevis	KN	KNA	Developing
Saint Lucia	LC	LCA	Developing
Saint Martin (French Part)	MF	MAF	Developing
Saint Vincent and the Grenadines	VC	VCT	Developing
Sint Maarten (Dutch part)	SX	SXM	Developing
Trinidad and Tobago	TT	TTO	Developing
Turks and Caicos Islands	TC	TCA	Developing
United States Virgin Islands	VI	VIR	Developing
Belize	BZ	BLZ	Developing
Costa Rica	CR	CRI	Developing
El Salvador	SV	SLV	Developing
Guatemala	GT	GTM	Developing
Honduras	HN	HND	Developing
Mexico	MX	MEX	Developing
Nicaragua	NI	NIC	Developing
Panama	PA	PAN	Developing
Argentina	AR	ARG	Developing
Bolivia (Plurinational State of)	BO	BOL	Developing
Bouvet Island	BV	BVT	Developing
Brazil	BR	BRA	Developing
Chile	CL	CHL	Developing
Colombia	CO	COL	Developing
Ecuador	EC	ECU	Developing
Falkland Islands (Malvinas)	FK	FLK	Developing
French Guiana	GF	GUF	Developing
Guyana	GY	GUY	Developing
Paraguay	PY	PRY	Developing
Peru	PE	PER	Developing
South Georgia and the South Sandwich Islands	GS	SGS	Developing
Suriname	SR	SUR	Developing
Uruguay	UY	URY	Developing
Venezuela (Bolivarian Republic of)	VE	VEN	Developing
Kazakhstan	KZ	KAZ	Developing
Kyrgyzstan	KG	KGZ	Developing
Tajikistan	TJ	TJK	Developing
Turkmenistan	TM	TKM	Developing
Uzbekistan	UZ	UZB	Developing
China	CN	CHN	Developing
China, Hong Kong Special Administrative Region	HK	HKG	Developing
China, Macao Special Administrative Region	MO	MAC	Developing
Democratic People’s Republic of Korea	KP	PRK	Developing
Mongolia	MN	MNG	Developing
Brunei Darussalam	BN	BRN	Developing
Cambodia	KH	KHM	Developing
Indonesia	ID	IDN	Developing
Lao People’s Democratic Republic	LA	LAO	Developing
Malaysia	MY	MYS	Developing
Myanmar	MM	MMR	Developing
Philippines	PH	PHL	Developing
Singapore	SG	SGP	Developing
Thailand	TH	THA	Developing
Timor-Leste	TL	TLS	Developing
Viet Nam	VN	VNM	Developing
Afghanistan	AF	AFG	Developing
Bangladesh	BD	BGD	Developing
Bhutan	BT	BTN	Developing
India	IN	IND	Developing
Iran (Islamic Republic of)	IR	IRN	Developing
Maldives	MV	MDV	Developing
Nepal	NP	NPL	Developing
Pakistan	PK	PAK	Developing
Sri Lanka	LK	LKA	Developing
Armenia	AM	ARM	Developing
Azerbaijan	AZ	AZE	Developing
Bahrain	BH	BHR	Developing
Georgia	GE	GEO	Developing
Iraq	IQ	IRQ	Developing
Jordan	JO	JOR	Developing
Kuwait	KW	KWT	Developing
Lebanon	LB	LBN	Developing
Oman	OM	OMN	Developing
Qatar	QA	QAT	Developing
Saudi Arabia	SA	SAU	Developing
State of Palestine	PS	PSE	Developing
Syrian Arab Republic	SY	SYR	Developing
Turkey	TR	TUR	Developing
United Arab Emirates	AE	ARE	Developing
Yemen	YE	YEM	Developing
Fiji	FJ	FJI	Developing
New Caledonia	NC	NCL	Developing
Papua New Guinea	PG	PNG	Developing
Solomon Islands	SB	SLB	Developing
Vanuatu	VU	VUT	Developing
Guam	GU	GUM	Developing
Kiribati	KI	KIR	Developing
Marshall Islands	MH	MHL	Developing
Micronesia (Federated States of)	FM	FSM	Developing
Nauru	NR	NRU	Developing
Northern Mariana Islands	MP	MNP	Developing
Palau	PW	PLW	Developing
United States Minor Outlying Islands	UM	UMI	Developing
American Samoa	AS	ASM	Developing
Cook Islands	CK	COK	Developing
French Polynesia	PF	PYF	Developing
Niue	NU	NIU	Developing
Pitcairn	PN	PCN	Developing
Samoa	WS	WSM	Developing
Tokelau	TK	TKL	Developing
Tonga	TO	TON	Developing
Tuvalu	TV	TUV	Developing
Wallis and Futuna Islands	WF	WLF	Developing

Adversarially (non-)robust machine learning