Neural Brushstroke Engine (NeuBE) includes a GAN model that learns to mimic many drawing media styles by learning from unlabeled images. The result is a GAN model that can be directly controlled by user strokes, with style code z corresponding to the style of the interactive brush (and not the final image). Together with a patch-based paining engine, NeuBE allows seamless drawing on a canvas of any size – in a wide variety of learned natural and novel AI brush styles. Our model can be trained on only about 200 images of drawing media, is shown to match the training styles well, and generalizes to unseen out-of-distribution styles. This invites novel applications, like text-based retrieval of brushes and matching brushes to existing art to allow interactive painting in that style. Our generator supports user control of stroke color for any brush style and compositing of strokes on clear background. We also support automatic stylization of line drawings.