I honestly don't mean to wipe trillions off the stock market but you can view a neural network layer as context choosing a linear mapping. Then BP comes along and makes that the best average linear mapping for that layer and context.
For a particular input a neural network is a composition of best average linear mappings.