Once your child is reading and beginning to do math (adding and subtracting)*, the only textbook you really need to buy is a math book. We use older versions of Saxon, because it is widely considered the best, but really, you can use whatever math book you like.
Then, if your grade school child enjoys doing worksheets, you might consider buying one of those inexpensive, grade appropriate worksbooks sold to supplement public school curriculum, especially during the summer. Or you could just find the many thousands of free worksheets available online, and print them out as needed.
Other than that, you just need access to lots of books. They could be library books, or they could be books from thrift stores. (I highly recommend St. Vincent dePaul's for books.) You can also buy new, of course, but I find that's rarely necessary.
So we have math covered, and reading covered. And everything else - from science to history - is also covered in the books your child reads (and/or you read to your child). This requires a little bit of forethought on the part of the parent-teacher, but it works really well. Now add to that science experiments, writing (which can include writing prompts and copywork related to whatever subject the child is studying at the moment), crafts, and other hands on activities - which you can easily find online - and you're all set!
So to be clear, all you need is:
* A good math textbook for your child's appropriate grade
* Lots of reading books
* And, if you like, a cheap general workbook for your child's appropriate grade
You child's spelling words come from her reading materials. Her history/social studies, science, and other school subjects also come from reading books - and doing experimenter and crafts you find for free online. Her writing comes from copying from reading books (for example, last year my daughter kept a journal of the Bill of Rights, which she also memorized) and getting writing assignments or prompts.
It is SO simple, and very affordable.
Want a little more hand holding? If you're brand new to homeschooling, you might also want to invest in (used) copies of What Your...Needs to Know books by E.D. Hirsch. These offer a guideline for what your child could be learning each year. I use them to determine - generally - what sort of books I want my children to read each year - what type of science, history, and so on.
Like the idea of focusing on math, writing, and reading, but want more hand holding? I recommend The Robinson Curriculum. For $195, you'll get CDs with all the curriculum you need through high school (minus math books), including reading material. Add a good laser printer, and you're all set for years! Be sure to read the history of this curriculum, and how all six of the Robinson children taught themselves with it; it's hard not to be impressed!
* If your child isn't reading yet, consider a book like Teach Your Child to Read in 100 Easy Lessons or Mommy, Teach Me to Read!, plus some good phonic readers (like the Bob books) to get him started. As for math, preschoolers and kindergarteners need to learn to count to 100 (it's nice to use a number chart for this), focus on memorizing addition facts (up to 10 or 12), followed by subtraction facts (also up to 10 or 12), using flash cards. Also, check out my posts on Homeschool Preschool.