I heard this question many times. I usually reply quickly in a few sentences, but I agree that OSGi deserves a more detailed answer.
Let's start with the basics. The Java Language Specification does not mention higher-level abstractions, like components. It follows the OO concepts, in which the highest level of abstraction is the class. Why is it important? There is a difference between a class and a component. A component specifies its provided and required interfaces, while a class is not able to do it. Of course, a class can contain public members that implicitly form a provided interface together (and we can extract them into a Java interface to make things more clear), but it is not a formal specification of a provided interface. Specifying the required interface is more problematic, it can only be extracted from the class using static analysis. It is far from being a formal specification, too.
This 'minor' weakness of the language makes real component-based development impossible without building an additional abstraction level at the top of classes and interfaces. We can call it a component model.
At the very beginning, Sun specified one called JavaBeans, but it was too simplistic and it was intended for building graphical user interfaces, not complex systems.
As Java became a mainstream language, complex tasks required sophisticated component models. It was the time when EJB was born. At the same time, many projects (Spring, PicoContainer, etc.) tried to address the same need in a lightweight fashion, but they all have two common problems: handling versioning and dynamic loading at component-level.
As the internet-boom was over, EJB lost popularity because of its complexity and price (application servers were very expensive at this time), and lightweight frameworks became more popular. OSGi was created at this time to address the same need, but it treated components as first-class entities (they called them bundles). This resulted in a complete component model that made component-based development possible. Furthermore, OSGi defines standard services, like logging, package management, etc. that simplifies the job of developers not wanting to reinvent the wheel.
There are some important consequences of having a good component model. First, a well-written component is reusable. With the introduction of component versioning, runtime library conflicts (JAR hell) can be eliminated. Second, component instances can provide services and this, together with dynamic component loading and component-level dependency management makes service composition possible.
To wrap it up: OSGi is a natural extension of Java, because of the nice component model and its dynamic nature. We really needed it for years when we were forced to manually handle all of the problems solved by OSGi in each and every project.