A simple wrapper for asynchronous file I/O (ReadFileEx,WriteFileEx)
Introduction
Hi all, here Iam dealing with a subject which is comparatively less discussed. The people new to programming world will start with samples which create, read or write files. In Windows all file related calls finally reach in CreateFile, ReadFile or WriteFile. We are all familiar with these APIs. Technically, ReadFile and WriteFile APIs are synchronous APIs. That is these APIs return only after the requested data is read or written. Now, coming to today’s topic. There are asynchronous versions of these APIs. The ReadFileEx and WriteFileEx. Actually ReadFile and WriteFile can also behave asynchronous. But our discussion is on the explicitly asynchronous APIs, ReadFilEx and WriteFilEx.
Background
Normally, when we need asynchronous behaviour or parallelism we create threads. The asynchronous file I/O APIs allows us to leverage the asynchronous behaviour without introducing threads. So we can issue ReadFileEx or WriteFileEx and perform other operations. Then finally when the application needs the result of I/O, it can check the status of asynchronous operation with the help of some APIs which we will discuss shortly. The file should be opened with FILE_FLAG_OVERLAPPED for using the asynchronous APIs.
See the prototype for ReadFileEx below. It is the same for WriteFileEx.
BOOL WINAPI ReadFileEx(
__in HANDLE hFile,
__out_opt LPVOID lpBuffer,
__in DWORD nNumberOfBytesToRead,
__inout LPOVERLAPPED lpOverlapped,
__in_opt LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);
Here we are interested in the last two parameters lpOverlapped and lpCompletionRoutine. We need to specify OVERLAPPED structure filled with offset from where data is to be read or written. There is a hEvent member in this structure. The MSDN documentation says that we are free to use this for our purpose. But there is asynchronous behaviour exhibited by ReadFile and WriteFile when its last parameter OVERLAPPED pointer is provided. In this case some MSDN documentation says that we need to supply separate event handles for each instance of OVERLAPPED structure. So there is a confusion that we need to specify the hEvent or not. Also it mentions that this event object will be signalled when the specified overlapped operation completes. The lpCompletionRoutine is the callback function to be called by the system when the specified asynchronous operation completes. It will be invoked per ReadFileEx/WriteFileEx call. There is a side track for this. This callback routine is invoked only when the thread enters in an “alertable” wait state. This state can be set with the help of SleepEx, WaitForSingleObjectEx, WaitForMultipleObjectsEx, MsgWaitForMultipleObjectsEx etc. A thread can issue asynchronous I/O operations and freely do other jobs. Then, once all its other tasks are completed, it can enter the alertable wait state. The callback routines are then called in the context of such APIs. The status of an overlapped operation can be checked with the help of GetOverlappedResult API.
Using the code
I have wrapped all these knowledge in to a small class AsyncFile.
class IAsyncOperationCompletionNotifier {
public:
virtual void OnAsyncOperationComplete(BOOL bRead,DWORD dwErrorCode)=0;
};
class AsyncFile{
public:
AsyncFile(LPCTSTR lpctszFileName,BOOL bCreate,DWORD dwDesiredAccess,
DWORD dwShareMode, IAsyncOperationCompletionNotifier* pINotifier,
BOOL bSequentialMode=FALSE, __int64 nStartOffset=0,
BOOL bInUIThread=FALSE);
BOOL IsOpen();
BOOL Write(LPVOID pvBuffer,DWORD dwBufLen,DWORD dwOffsetLow=0,DWORD dwOffsetHigh=0);
BOOL Read(LPVOID pvBuffer,DWORD dwBufLen,DWORD dwOffsetLow=0,DWORD dwOffsetHigh=0);
BOOL IsAsyncIOComplete(BOOL bFlushBuffers=TRUE);
BOOL AbortIO();
DWORD GetFileLength(DWORD* pdwOffsetHigh);
__int64 GetLargeFileLength();
VOID Reset(BOOL bSequentialMode=FALSE,__int64 nStartOffset=0);
operator HANDLE()
{
return m_hAsyncFile;
}
BOOL SeekFile(__int64 nBytesToSeek,__int64& nNewOffset,DWORD dwSeekOption);
~AsyncFile(void);
private:
....
};
The class object can be created specifying the file name, whether need to create new file, the access mode (READ/WRITE), share mode (same as CreateFile), callback interface to register for I/O completion notifications, whether sequential mode or not, the start offset and finally whether the thread is hosting a UI or not. If UI thread, the MsgWaitForMultipleObjectEx API is used for I/O completion wait (to set alertable wait state). In the case of non-UI thread, WaitForSingleObjectEx is used. An event m_hIOCompleteEvent is used to synchronize the wait. It is a manual reset event and it should be as per MSDN documentation. This is because; if the event is auto-reset the wait functions can alter its state. This can cause un-intended deadlocks if call GetOverlappedResult with its wait for completion flag as TRUE. The m_hIOCompleteEvent is set when the completion routines are invoked for each asynchronous request made. Actually a counter is kept for each asynchronous call issued. This counter is decremented when the completion routine is invoked. When it reaches zero the event is signalled.
The Read/Write functions perform overlapped I/O operations. If the sequential mode flag is TRUE, the offset will be incremented by the buffer length requested to read/write. After the Read/Write requests are issued IsAsyncIOComplete function can be called to trigger the alertable wait state and get queued I/O completion routines to be executed. This function checks the status of overlapped operations by calling GetOverlappedResult for each asynchronous operation requested. Also if the bFlushBuffers flag is TRUE, it will flush the file buffers to disk with the help of FlushFileBuffers API. This may be necessary, because the system writes the data to an intermediate cache for performance. These cached contents are flushed to disk on a later time. Such a mechanism improves the application performance, but the risk of data loss (if an unexpected shutdown occurs due to power failure or so).
The asynchronous I/O operations can be aborted by issuing CancelIo API. This will be success only if the API is called by the same thread which performs the asynchronous I/O. The functions are well commented.
Also I have prepared a demo application (VS2010 solution and VC6.0 dsp provided). This application accepts a source file and copies the contents to a target file. It use an instance of AsyncFile to read the source file and another instance to write to the target file. The progress of copy is visible in the UI. Also the user can abort the operation in between. I have tested the same in Win7 and to some extend in XP. The debug trace can be seen in DebugView. Thatz all for now! Thank you!
Points of Interest
As I mentioned in the beginning, the asynchronous file I/O is not a well discussed topic. So the design of AsyncFile class is fixed after some experiment with these APIs. I have tried with more than 700 MB file (got worked!). In the demo application, I have used one instance of AsyncFile to read and another instance to write. I have not tried the same instance for read and write. Looking forward to see how this class is being used.
History
发表评论
RjCw1v Just wanna input that you have a very nice web site , I like the layout it really stands out.
P0LMAl Looking forward to reading more. Great post.Really looking forward to read more. Cool.
mjXTHH Wow, this paragraph is fastidious, my younger sister is analyzing such things, therefore I am going to tell her.
OXsmHT phase I take care of such information a lot. I used to be seeking this certain info for a long time.
Just Browsing While I was browsing yesterday I noticed a great post about
RBizr3 that аАабТТаАабТа certаА аЂаinly much toаА аБТ big or small
tRg0Rn I really liked your article post.Thanks Again. Really Great.
Very nice blog post. I definitely appreciate this website.
Thanks! Fotbollströjor BarnI got this web page from my friend who told me on the topic of this site
and now this time I am browsing this web site and reading very informative articles or reviews here.
Inter TröjaGreat site. A lot of helpful information here. I'm sending
it to a few friends ans additionally sharing in delicious.
And naturally, thanks in your effort! Billiga Lazio FotbollströjaI want to to thank you for this wonderful read!!
Pretty! This was an incredibly wonderful post. Thank you for providing this info.
I'аve read several just right stuff here. Certainly price bookmarking for revisiting. I wonder how a lot effort you set to create such a fantastic informative web site.
Thank you ever so for you article post.Really looking forward to read more. Really Cool.
There is certainly a lot to know about this subject. I like all of the points you made.
Some really quality articles on this site, saved to bookmarks.
in the early hours in the dawn, because i love to gain knowledge of more and more.
You could definitely see your enthusiasm in the work you write. The arena hopes for more passionate writers such as you who aren at afraid to say how they believe. At all times follow your heart.
Utterly indited subject material, Really enjoyed studying.
Wow, great blog article.Really thank you! Cool.
Wow, incredible blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your web site is great, let alone the content!
There is perceptibly a bunch to identify about this. I suppose you made some nice points in features also.
I really liked your post.Much thanks again. Awesome.
romance understanding. With online video clip clip
Many thanks for sharing this fine piece. Very interesting ideas! (as always, btw)
The entire look of your site is fantastic, let neatly as the content!
Utterly written subject material, appreciate it for selective information.
Im thankful for the blog.Much thanks again. Great.
Say, you got a nice blog article.Really thank you! Awesome. ventolin
There is clearly a lot to realize about this. I assume you made various nice points in features also.
That is a beautiful picture with very good light -)
Thanks a lot for the article post.Really looking forward to read more. Really Cool.
Pretty! This has been an extremely wonderful post. Thank you for providing this info.
is excellent but with pics and videos, this website could undeniably be one of
Its hard to find good help I am constantnly saying that its hard to find good help, but here is
This very blog is no doubt interesting and also factual. I have found many handy things out of this amazing blog. I ad love to come back again and again. Thanks a lot!
With havin so much content do you ever run into any problems of
plagorism or copyright infringement? My site has a lot of completely unique content I've either created myself or outsourced but it seems
a lot of it is popping it up all over the internet without my permission. Do you know any methods to help protect against content
from being stolen? I'd really appreciate it. magliette NapoliI saw plenty of website but I conceive this one contains a thing special in it. The finest effect regarding fine people is experienced after we ave got left their presence. by Rob Waldo Emerson.
Im thankful for the blog article.Much thanks again. Much obliged.
Very energetic post, I loved that a lot. Will there be a
part 2? Billige juventus fodboldtrøjerMuchos Gracias for your article post.Thanks Again. Great.
Perfect piece of work you have done, this internet site is really cool with excellent information.
Major thanks for the post.Thanks Again. Really Great.
Wow! This can be one particular of the most useful blogs We have ever arrive across on this subject. Basically Excellent. I am also a specialist in this topic so I can understand your hard work.
Really appreciate you sharing this article.Really thank you! Will read on...
Spot on with this write-up, I really believe this amazing site needs a great deal more attention. I all probably be returning to read more, thanks for the info!
JMslDC This content has a lot of great information that is apparently intended to make you think. There are excellent points made here and I agree on many. I like the way this content is written.
wow, awesome article.Thanks Again. Will read on
papers but now as I am a user of net so from now I am
Hello to every body, it's my first pay a quick visit of this website; this blog carries awesome and actually good
data designed for readers. Maglie Calcio Bambini