I think that Rabbit is right in that cultivating a good writer's intuition is essential to writing great fiction (some of the best things I've written came straight from my gut without all the "rules" I learnt getting in the way), and that's something you can develop by simply reading and writing and living and feeling as much as you can.
But you knew that already. And for most of us non-geniuses, that's not quite enough.
Books on writing can definitely help you improve your craft. Don't think of it as learning rigid "rules". Writing is a skill, and like any other skill, it can be honed by learning from the insights of masters who have figured out what (in their opinion) makes good writing good. It's why writers should learn to analyze and critique literature in great detail; by trying to figure out "how fiction works", essentially, you increase your knowledge of the tools of the writing craft and how to use them more efficiently. By integrating this kind of technical writing knowledge with your own reading and writing experiences, you learn how those tools work for
you. That Mark Twain list, for example, has some gems in it, but that's obviously a list that works for Mark Twain and writers like him. You'll see great works of fiction breaking rules from that list constantly. And also it's rather simplistic. There are many, many theories and opinions out there on how to write well; I recommend that you read advice from a wide variety of authors that you like and respect and try and figure out how to best incorporate what you've learnt into your own work, instead of just sticking to a simple list of rules.
Here are some recommendations:
This recent Writers Unboxed
article seems like it would be very useful to you. Pay attention to the three key points she brings up at the end of the interview. I think they might be crucial to solving some of the issues you say you have. Of course, this is merely an article, and not a full on book, but it might be a good place to start.
I also highly recommend Stephen King's
On Writing, which is a highly entertaining piece of writing in its own right, in addition to being a great guide on how to write well. Another book that was quite useful to me personally is Nancy Kress's
Dynamic Characters, which focuses on the relationship between character and plot and how you can generate gripping stories with the right kind of characterization.
Just today, Writer's Digest put out a tutorial that seems like exactly the kind of thing you're looking for:
An Agent's Tips on Story Structures that Sell, which also has book recommendations as well. It's from a literary agent's perspective, so that might be even more useful, since you'll get the perspective of somebody who gets books published. (Writer's Digest is a great resource for writers in general, so it may be worth your time searching the site for articles/posts that have to do with your issues.)
And last but not least, try
Plot & Structure (subtitle:
Techniques And Exercises For Crafting A Plot That Grips Readers From Start To Finish) by James Scott Bell. This book seems like it would be the most directly useful to you, what with its emphasis on everything to do with plots. Here's a review excerpt from Amazon:
Bell addresses just about every aspect of plotting I could think of, from "What's a Plot, Anyway?" to generating ideas, dealing with beginnings (and middles, and endings), handling individual scenes, crafting complex plots, integrating character arc into plot, different systems of crafting plot, revising plots, plot patterns, plot problems, cures for plot problems, and even checklists to go through to make sure you're remembering everything as you write your book.
In some chapters, he does go into specific tips for genre novels, but he pays attention to literary fiction as well, and most of what he talks about can be applied to both genre and literary fiction.
How NOT to Write a Novel was a hilarious read, and if you want some light, funny reading, I would highly recommend it, but for some reason the stuff it talked about just didn't stick with me. I don't think I remember a single tip from it. Maybe that's just me though.