The idea of disobedient machines is developed from the perspective of the historical and current developments in artificial intelligence (AI). Disobedience is often used in arts and technology as both a theme and a tool. Beyond that, misbehaviour is presented as one of the skills that is indispensable for natural intelligence. The article doesn’t delve into the use of AIs as an assistive tool for creation. Instead, it speculates if AIs will afford the emergence of an independent, autonomous artificial creator. Different approaches to AIs are presented, from symbolism to emergism. The affordances of machine learning models are described, as well as their limitations like the incapacity to generate breakthroughs outside of their training data, their determinism, and the inability to use analogies to solve unseen problems. Other missing human (or biological) skills present in art are emotion, goal-less production, and agency, which is a problem even when human volition is studied. The limits of computational formalism are like the limits in mathematical reasoning – it always requires some external rules, or axioms demonstrated, like Gödel’s proof. Hofsdtader’s theory of consciousness proposes a way to conciliate the fact that human creativity is also based on closed, fixed biological rules. Finally, it is argued that a machine cannot be creative unless it is also able to misbehave. However, computers must follow a set of instructions or they stop functioning – that is the definition of a Turing machine. Hence, we must face the paradox of wanting well-behaved systems, with the limitations of symbolic machines, while at the same time demanding more autonomous, creative outputs. It is paramount to explore algorithmic misbehaviours that could circumvent this paradox for further development of AIs for the arts and society in general.