If you wish to use the Arduino as the main part of your ESP8266 project, there are several ways to go.
- Use the Arduino to issue AT commands to the ESP8266.
- Create your own protocol or messaging system by programming up the
ESP8266 with (for example) nodemcu or the Arduino/ESP8266 project or
Espressif or others, then program up your Arduino with the same
protocol or messaging system so the two may talk.
Alternatively, just program your ESP8266 directly. No Arduino required. Doing so, you can turn the tables and get the ESP8266 to issue messages to the Arduino if for example if you wanted the many GPIO and sense pins of the Arduino to do something. If you only need a couple of extra GPIO pins, look at the ESP8266 range to get more GPIO pins, such as the ESP8266-12
Which to choose?
- nodemcu will give you a good idea of the capabilities of the ESP8266
and may be a good starter, easy to flash, easy to program, but it is a poor finisher except for the most basic of programs.
- espressif has a large toolchain, not pleasant to flash,
you'll be at the cutting edge, however the community support is
minimal
- The Arduino/ESP8266 project is awesome, easy to flash, very fast,
very stable, and unlike nodemcu you can create a large project. As you noticed, any arduino project starting with
#include <ESP8266WiFi.h> is not for the arduino, but for the ESP8266 using the arduino IDE https://github.com/esp8266/Arduino