No matter what form of electronic music one records, something to record on is absolutely essential. A DAW is short for a digital audio workstation, and is the best way to record music. A DAW uses a personal computer to record audio, it is also a host for plugin software synthesizers and effects.
My advice is to start out by getting an inexpensive DAW. For those with MACs, Garageband will do the trick. For those who are starting on a PC or Linux, Reaper or Tracktion are likely the best option for a low cost platform.
There are a large number of DAWs and everybody has their favorite. If you do research and find something you can both afford and fits what seems like you would like to use and learn it. I only mention Reaper and Tracktion because of their affordability. My advice is to make sure the DAW is compatible with the native platform for plugins for your mac or pc. For Macs this is the AU format, for Windows and Linux this is VST. A word of warning though, not all VST plugins or hardware can be used with Linux. While Linux is becoming more common in terms of compatibility.
Since each DAW is a little unique in terms of functionality, I highly suggest you watch and read the tutorials and manuals for the specific one that you will be using. I personally use Ableton Live, it is a great piece of software, and a lite version often comes with many controllers, which are either keyboards, drum pads, or other physical devices to control software. If you chose a controller that comes with Ableton lite, it is also a more than appropriate DAW and host to get started. With that being said one does not need to start on with Ableton Live and Ableton Live Suite, both of which carry a high cost. Ableton Live is a wonderful tool for those who are experienced, but the more expensive software out there is not necessary to get started.
A USB Midi controller is a nice thing to have when starting out. Smaller ones are easy to find both new and used that would likely be appropriate. Controllers can range from relatively inexpensive keyboards, to esoteric ones such as the Eigenharp which cost thousands of dollars. I highly suggest just getting a decent inexpensive keyboard. If you want to go for both the hardware synth and software synth route though, something like the arturia microbrute can also be used as a controller.
All DAWs also work with audio interfaces which work with either USB or Firewire. This will allow you to record audio signals using either lines in or microphones. This however is not needed for doing music with plugin synthesizers. However, if you do move onto hardware synthesis or incorporating musique concrete it is recommended to get some type of audio interface. The microphone plug into your computer is generally not sufficient to record high quality audio.
The one thing I do not recommend to get started is Pro-Tools or Reason. While these platforms are frequently pushed for a synthesist the insistence of an exclusive plugin format is problematic for both. They do not take Native plugins as a result there are barriers to some of the best software instruments with these platforms. Other than that most DAWs will do the same job, but in slightly different ways with different interfaces and features. But outside of the exceptions, most can host native software instrument and effects plugins on most DAW platforms.
Once you have chosen your DAW and have installed it on your computer, the next step is to identify some soft synth plugins, and start to record music.