Monday, May 22. 2006Filtering & Escaping Cheat Sheet
After having two conversation on escaping data for !MySQL insertion within 5 minutes of each other in ##php@freenode, I decided to created a cheat sheet on Filtering & Escaping in general. I think what most people are not aware of is that not just outputting to the browser is output. So it querying a database, or calling one of the exec functions, or even using the mail function (not mentioned on the cheat sheet, perhaps for v2 Filtering & Escaping Cheat Sheet (PDF) - Grab it now! Anyhow, I hope people find this useful! - Davey Update: This is not just for !MySQL users, everybody should read it, I provided alternates to !MySQL's methods in the on place I use them Update 2: Digg It Update 3: Fixed the typo Comments
Display comments as
(Linear | Threaded)
This may not be the appropriate place to raise this question, but since you are on the subject, i have a tricky one for you. (And maybe its just late and my brain isn't working properly.)
On the cheat sheet you have most alpha/numeric input going through the ctype functions, however, what would you do in the case of many names that are hyphenated or contain apostrophes?
It must be late... I just realized that I typed my own name wrong in my email address.
I think this is a good sheet, but I'm not with your opinion.
The validation/filtering-part is in most cases work, which is not needed. Some mantras: - Filtering/validating the user input does not solve security problems! [1] - Escaping the user input will not help you out from escape-hell - When a user wants to store his ZIP-Code as "FOOOOOO", why not? Perhaps he comes from a strange country, where ZIP-Codes are not numbers and it's not my problem, when he stores the wrong ZIP-code. To [1]: Security problems are for the 99,999% of all cases, when you OUTPUT DATA! So no matter how you are escaping/filtering/validating incoming data: The problem is the OUTPUT. This part of the sheet is very good. Some more examples: - Why must a ZIP-code be a number? For example in Germany there are ZIP-Numbers, begining with 0. Other countries take letters in the ZIP. And so on. I store a ZIP-Code always as string(10) or so. - Why should I escape HTML, before I store it? It never can't be restored to it's original. Instead of that, I take the data as that, what it is: A string, which can be stored in the DB or been output as escaped HTML. Again: Escaping incoming data doesn't give you any more security, cause you cannot know, in which context you output the data. Some examples, where validation makes sense: Credit card numbers. Phone-numbers: I remove only letters . A very light sort of "this looks like an email-address": looks like "foo@ba.r". I try to take the user input as it's original input as long as it works. I try to take the data as what it is: A string. The more flexible is my program at the end. I avoid converting functions, which can't be reverted, before I store my data, cause that leads to those funny programs, which prints out things like "Alex\'s Dream" instead of "Alex's Dream". So - finaly - I would add your chart another path: Add a new path from the "user-input" directly to the "validated data"-cloud and write "Original string data". What I'm also missing is, that you are focusing only to the user-data-input. Databases are also input. I just mention this.
Quoting: Again: Escaping incoming data doesn't give you any more security, cause you cannot know, in which context you output the data.
Filtering/Validating data is a long recommended and necessary step in web applications where the input (from the User) is considered to be untrustworthy in all interpretations of the word. Escaping is done prior to outputting data to a browser, database, or other output channel. Er, on context. Isn't that what you decide when you start pushing data into the database and html output? Just wondering... Stating one half of the equation is not required is foolish. What if the user inputs invalid data and I proceed to use it without bothering to filter it, or validate the contents? Don't you think this might lead to a few embarrassing security exploits? All that bad data floating around my application having a free reign to interfere with my business logic which assumes it's totally valid... Even escaping would not escape all those unfiltered hacks to outwit escaping functions. Bad idea. Davey's PDF cheat sheet is a perfect summation of user data filtering and escaping. I've posted the link as a security resource to the Devnetwork forums and I'm sure it will help many a developer seeking security advice and help. Another addition to the poster collection above my monitor... .
Alex, your primary complaint seems to be that input should not be escaped, which is true. However, I can only see one mistake Davey made in that regard - rawurlencode() is described as filtering (it's escaping). Are there other instances that I'm missing, or are you misinterpreting the flow?
Also, please don't think that escaping output solves all of your problems. Even FIEO doesn't protect you from everything. It's naive to think otherwise. Some comments on the cheatsheet: 1. I'd drop the arbitrary distinction between validation and filtering. 2. The entire filtering cloud can be dropped or replaced. At the very least, stripslashes(), urldecode(), and rawurlencode() are not useful for filtering. 3. You could drop HTTP from the URLs & HTTP cloud, since you don't indicate anything for HTTP (plus, HTTP is an entire protocol and not a specific context). 4. It would be nice to reverse the order of the HTML & XML cloud and the URLs & HTTP cloud, because many developers have trouble properly escaping links. Anyway, great job on the cheatsheet - I'm sure it will help a lot of people. 
The reason I call it URLs & HTTP is you should use the urlencode/rawurlencode methods in GET/POST args for HTTP (i.e. for XMLHttpRequests).
I will go about maving v2 with all the input, thanks  - Davey
Good stuff davey. I wish I would have had this a year ago when I first started learning PHP.
|
About Me
Davey Shafik is a full time PHP Developer with 7-10 years experience in PHP and related technologies. An avid magazine writer and soon to be book author, Davey keeps his mind sharp by trying to tackle problems from a unique perspective.
ConferencesNone at this time! Calendar
QuicksearchTagsArchivesSyndicate This BlogBlog Administration |
|||||||||||||||||||||||||||||||||||||||||||||||||
Davey of the Pixelated Dreams blog have made a pretty neat chart of how you can deal with filtering and escaping of input data. The chart can be located here and the article here. Gives a decent overview of what php functionality are used to do filteri
Tracked: May 23, 10:27
Davey Shafik has posted a Filtering and Escaping Cheat Sheet for PHP. It's a neat, one page cheat sheet, with the filtering/escaping basics laid out and organised. I won't say it will make you a security expert in a day (it's a cheat sheet afterall!) b
Tracked: May 23, 13:06
After having at last the time to follow my news subscriptions I found some good articles to bookmark for later investigation. Have a look insideTwo parts article on PHP + AJAX:Part 1 Getting Started walks you getting setup and building a simple photo alb
Tracked: Jun 17, 17:01