I have the following code:
for item in soup.select('.profile-detail'):
f= open('htmlfile.html', 'w')
f.write(item)
f.close()
I want to write items into "htmlfile.html" but it's giving me the error:
TypeError: write() argument must be str, not Tag
Just use str() to get the whole tag content :
with open('htmlfile.html', 'w') as f:
for item in soup.select('.profile-detail'):
f.write(str(item) + '\n') # thanks jeteon :p, it's cleaner
I gather from the soup in your snipped that you're using BeautifulSoup to extract elements with the class "profile-detail". Given that, there are two problems with the code you have:
The value item in the list returned by the select() function is an instance of the Tag class and the the write method of the file object expects a string. As @PRMoureu wrote, you can cast the Tag instance to a string and it'll return the raw HTML string it represents by replacing the file writing line with this:
f.write(str(item))
The file you've open is opened in write ("w") mode in the loop. This means that for each iteration of the loop the file would be overwritten and if you were trying to collect all the elements returned by your query, you would only get the last one. If you want to have all of them in the file, the following are alternative ways to fix it:
# Open the file in append mode
for item in soup.select('.profile-detail'):
f = open('htmlfile.html', 'a')
f.write(item)
f.close()
This isn't too good though since it is unnecessary to open and close the file on each iteration. We can keep the file open for all our writes:
# Open and close file just once
f = open('htmlfile.html', 'w') # Okay to open in write mode now
for item in soup.select('.profile-detail'):
f.write(item)
f.write('\n') # In case you're expecting each on a new line
f.close()
Or, my personal favourite, doing pretty much the same but with contexts so you don't worry about forgetting the f.close() or accidentally giving it the wrong indentation or something later:
# With contexts
with open('htmlfile.html', 'w') as f:
for item in soup.select('.profile-detail'):
f.write(item)
f.write('\n') # In case you're expecting each on a new line
# File is auto-magically closed by the time you get here