No, you don't have to include all of POI's dependencies. Maven's transitive dependency mechanism will take care of that. As noted you just have to express a dependency on the appropriate POI artifact. For example:
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.8-beta4</version>
</dependency>
Edit(UPDATE): I don't know about previous versions but to resolve imports to XSSFWorkbook and other classes in org.apache.poi package you need to add dependency for poi-ooxml too. The dependencies will be:
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>4.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>4.1.2</version>
</dependency>
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…