I've moved!

Hi

I recently joined forces with an old friend of mine and opened up a blog together.
We figured in these times... good people should stick together.

Anyway, our blog is at http://www.morethantechnical.com

It's going to be about the same stuff you see here (In fact I copied all my posts there), programming, cool stuff I come across my day, comics and artistic things also..

My friend, Arnon, also brings a lot to the table. He's more of a get-things-done kind of guy, and somewhat of a hacker, I know you'll like his stuff. We go a long way back.

OK,
So I hope to see you there!
And don't be shy to comment...

Roy.

So you're trying to get your homemade app on your pwned iPhone

Recently I was working on an iPhone app for work, for demo purposes, but my company cheaped out on the Apple iPhone Developers registration. 

More accurately, the process of binding a multi-million-$-a-year company with Apple Inc. takes a very long time, so we took the back-door and just pwned our test iPhone devices (firmware 2.1).

"How hard can it possibly be to install apps on a jail-broken iPhone?" we thought, Well as it turns out, it's pretty difficult, especially for Mac-first-timers like myself. 

In the end, I overcame this obstacle - but not before compiling a compiler, installing a gazillion support apps, compiling my app with at least 6 different compilers, doing it on WinXP, Ubuntu, on the iPhone itself, and on the Mac.

So I thought why not share with you the way that actually produced the working result.

Well I got a working app using 2 compilers, and only on the Mac machine (intel, Mas OS 10.5.4):

• Open Toolchain:  LLVM-based compiler, which you need to compile from scratch. Instructions can be found here.
• The ARM compiler that came with the iPhone SDK itself.

(but I hear that compiling under WinXP with Cygwin, which I also tried and gave up on,  can actually work)

First thing I tried, naturally, was to try and use XCode's internal functionality of deploying & debugging on the device, but that requires code signing, and to get a valid certificate - you must pay. Apple: 1 - Roy: 0. 

So I went to find a solution, and for a very long time I was convinced it was too hard for a total Mac beginner. But as the due date of my app was getting closer, and still we weren't legally registered with Apple's dev. program, I started to look more closely at deploying the app manually.

I found iphone-dev, which is an alternative to xcode's compiler. At the time I thought that the SDK's compiler was the problem. So I followed all the instructions on the Wiki and soon enough I had a working GNU c compiler: arm-apple-darwin-gcc.

And that compiler was very successful at compiling the HelloWorld apps, even ones I had made with XCode templates! I was very pleased. Apple:1 - Roy:1!!

Few things I had learned though:

•  XIB files must be compiled into NIB files before they are bundled up together with the rest of the app.
  

It's done with iPhone SDK's 'ibtool'. So I added this into my Makefile:

compilexib: $(XIBS) 

%.nib: %.xib

ibtool --errors --warnings --notices --output-format human-readable-text --compile $@ $<

• The compiler doesn't handle property assigning very well. It doesn't support this method of assiging: myObj.someprop = someVal;, instead you must use the Objective-C way of: [myObj setSomeProp: someVal];.
• The executable must be signed somehow, but as the device jailbreaking frees it from complying to Apple's code signing conventions, you can sign it any way you want. Easiest way: "ldid" from Saurik's house. Download "ldid" from Cydia and follow the instructions.

Now, the biggest problem I had with my HelloWorld apps was linking. Compiling went fine in most cases, and in case of problems the compiler notifies in a somewhat comprehendible way what is the problem so it can be fixed. But the linker, it obviously has a very different notion of what is an understandable error message. And I found myself just trying different permutations of the LDFLAGS in my Makefile until I got it right. This is what I finally came up with:

LDFLAGS = -lobjc -framework CoreFoundation -framework Foundation -framework UIKit -framework CoreGraphics -framework CoreLocation

LDFLAGS += -L"/usr/lib"

This is actually all you need for open toolchain's linker.

 CFLAGS should be:

CFLAGS = -fobjc-abi-version=2 -lobjc

CFLAGS += -I"/var/include/"

CFLAGS += -I"/usr/include/"

CFLAGS += -I"/usr/local/arm-apple-darwin/include"

I used /var/include to softlink the frameworks I used in my project:

lrwxr-xr-x  1 root  wheel  130 Jan 20 14:59 CoreFoundation -> /Developer/Platforms/iPhoneOS.platform//Developer/SDKs/iPhoneOS2.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/

But this compiler is not without its own shortcomings. It's based on the Mac OS 10.4 SDK, which apparently has no GC in it, so when I tried to compile my actual app I got this linker error:

undefined symbols: _objc_assign_ivar

A wretched error, barely any information about it online. And it was the only error in the build, so you can imagine my frustration at that stage. But a fishing tour around Google revealed that, as I mentioned, the problem is that OSX 10.4 SDK doesn't support some function that was supported by 10.5 (the environment I was developing on in XCode).

I had no intention to look around every corner of my code to find these spots where theres a problem, and anyway since this is a linker problem I had no way of finding these spots.

Ok, so then I thought - iphone-dev's compiler is a bust, I gotta give that SDK compiler another go. And I did, and sure enough it worked. But only after some tinkering, of course.

First off, a different compiler and its own headers:

CC=arm-apple-darwin9-gcc-4.0.1

CFLAGS = -fobjc-abi-version=2

CFLAGS += -I"/var/include/"

CFLAGS += -I"/Developer/Platforms/iPhoneOS.platform/Developer/usr/include"

CFLAGS += -I"/Developer/Platforms/iPhoneOS.platform/Developer/usr/lib/gcc/arm-apple-darwin9/4.0.1/include"

CFLAGS += -I"/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.0.sdk/usr/include"

And the linker needs its own libraries to link to:

LDFLAGS = -lobjc -framework CoreFoundation -framework Foundation -framework UIKit -framework CoreGraphics -framework CoreLocation

LDFLAGS += -L"/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.0.sdk/usr/lib"

LDFLAGS += -F"/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.0.sdk/System/Library/Frameworks"

But after all that was set, took me a while to find all the depenencies - I was good to go!

My application, including UI built with Interface Builder, was compiling and running smoothly on the pwned device.

All you need is to put your executable, resources and Info.plist in a *.app directory, and send that app to your iPhone's /Applications/ directory (I used plain old scp).

To ease my work on the device I used BossPrefs and OpenSSH from Cydia.

BossPrefs has a nice function of "Fast respring" in the "Power" menu, that will reindex the /Applications directory and find your app there.

Another last tip: tail the syslog, download "syslogd to /var/log/syslog" from cydia and "tail -f /var/log/syslog".

I hope this was helpful.

Good luck building your own iPhone apps!

Roy.

OpenGL for AviSynth

Hi

I had a little project at work recently, that involved creating movie clips using AviSynth.
And I was appalled by the shabbiness of existing transition plugins available freely for AviSynth, they always reminded me of 80s-like video editing...
So I set out to integrate AviSynth with OpenGL to create a nice 3D transition effect for our movie clips.

I had 2 major bases to cover:

• AviSynth plugin API
• OpenGL rendering

AviSynth API is not so well documented, but they have very good ground-up examples on how to DIY plugin. Here is the one I used, that basically does nothing but copy the input frame to the output frame.
Open GL on the other hand is very well documented and "tutorialed". I based my code on this example from NeHe.

So basically what I wanted to achive is:

1. Read input frame (AviSynth)
   
2. Paint frame as texture over 3D model (OpenGL)
3. Draw rendered 3D image to output frame (OpenGL+AviSynth)

Reading the frame is pretty straightforward. Frames come encoded as RGB 24bit, with a little twist: rows size in bytes is not width*3 as you'd expect it be, but AviSynth use a parameter called "Pitch" to determine row size in bytes.

So when I extract the input image I do:

uchar* texBuf = (uchar*)malloc(width * height * 3); //"pure" RGB24 encoding
const unsigned char* srcp = src->GetReadPtr(); //strange AviSynth RGB24 encoded data

memset(texBuf,0,width*height*3);
int line_length = width*3;
for(int y=0;y [lt] height; y++)
const unsigned char* line_srcp = srcp + y*src_pitch;
unsigned char* line_dstp = texBuf + (height - 1 - y)*line_length;
memcpy(line_dstp,line_srcp,line_length);
}


OK, now I have the input frame in my memory space and I can move on to make an OpenGL texture out of it:

glGenTextures(1,&tex);
glBindTexture(GL_TEXTURE_2D, tex);
gluBuild2DMipmaps( GL_TEXTURE_2D, 3, width,height,
GL_RGB, GL_UNSIGNED_BYTE, texBuf );

Cool.

This is when I ran into the first wall. In order to render anything, OpenGL must have a rendering context, for a rendering context you need an OS drawing context. In Win32 case a drawing context can be either a bona fide actual window, or a memory-based BITMAP...

I burned many hours trying to make BITMAPs work as offscreen redering contexts for OpenGL but with no avail... So I went with the "dirtier" solution of creating a window for the sake of off-screen redering (which kinda takes the "off" out of "offscreen"), but it works..
For easier setup I used GLUT (with an OpenGLUT impl.) and GLEE:

glutInitDisplayMode ( GLUT_RGB | GLUT_DOUBLE );
glutInitWindowSize(width,height);
glutCreateWindow( "offscreen" );

glutDisplayFunc ( display );
glutReshapeFunc ( reshape );
glutKeyboardFunc ( keyboard );
glutIdleFunc ( idle );

GLeeInit();

This is done in each frame, sadly, but I wasn't able to make it work otherwise.
Now, another hack I had to do is have the GLUT main loop render only one frame... Because the whole thing is setup again in the next frame.
So I enter the main loop after all settings are done:

glutMainLoop();

And in the end of the display() function I exit the main loop: (this can only be done in OpenGLUT, regular GLUT does not support this)

glFlush();
glFinish();

glReadPixels(0,0,width,height,GL_RGB,GL_UNSIGNED_BYTE,img_data);

glutLeaveMainLoop();

But, not before I grab all the pixel data from the rendered image using glReadPixels. This will go into the output frame of the AviSynth session:

for(int y=0;y [lt] height; y++)
unsigned char* line_dstp = dstp + y*dst_pitch;
unsigned char* line_srcp = img_data + y*width*3;
memcpy(line_dstp,line_srcp,line_length);
}


OK, we're pretty much done. Just need to draw some polygons in the display() function, use the tex texture and the input movie will appear frame-by-frame on the polygons - sweet!

Some results
This is the .avs file:
LoadPlugin("opengl-avisynth.dll")
A = ImageSource("A.jpg",end=70)
B = ImageSource("B.jpg",end=50)
Dissolve(A,B,20).SimpleSample

This is the result:


Another result, this time with actual video as an input:
A = AVISource("MVI_6130.avi.AVI").ConvertToRGB24()
SimpleSample(A)

Lay-off badges

The Israeli hi-tech market inspired me...

Showing video with Qt toolbox and ffmpeg libraries

I recently had to build a demo client that shows short video messages for Ubuntu environment.
After checking out GTK+ I decided to go with the more natively OOP Qt toolbox (GTKmm didn't look right to me), and I think i made the right choice.

So anyway, I have my video files encoded in some unknown format and I need my program to show them in a some widget. I went around looking for an exiting example, but i couldn't find anything concrete, except for a good tip here that led me here for an example of using ffmpeg's libavformat and libavcodec, but no end-to-end example including the Qt code.

The ffmpeg example was simple enough to just copy-paste into my project, but the whole painting over the widget's canvas was not covered. Turns out painting video is not as simple as overriding paintEvent()...

Firstly, you need a separate thread for grabbing frames from the video file, because you won't let the GUI event thread do that.
That makes sense, but when the frame-grabbing thread (I called VideoThread) actually grabbed a frame and inserted it somewhere in the memory, I needed to tell the GUI thread to take that buffered pixels and paint them over the widget's canvas.

This is the moment where I praise Qt's excellent Signals/Slots mechanism. So I'll have my VideoThread emit a signal notifying some external entity that a new frame is in the buffer.
Here's a little code:

void VideoThread::run() {
/*
... Initialize libavformat & libavcodec data structures.
You can see it in the example i referred to before
*/

// Open video file
if(av_open_input_file(&pFormatCtx,
"lala.avi",
NULL, 0, NULL)!=0)
return -1; // Couldn't open file

// Retrieve stream information
if(av_find_stream_info(pFormatCtx)<0)
return -1; // Couldn't find stream information

// Find the first video stream ...

// Get a pointer to the codec context for the video
// stream...

// Find the decoder for the video stream...

// Open codec...

// Allocate video frame
pFrame=avcodec_alloc_frame();

// Allocate an AVFrame structure
pFrameRGB=avcodec_alloc_frame();
if(pFrameRGB==NULL)
return -1;

int dst_fmt = PIX_FMT_RGB24;
int dst_w = 160;
int dst_h = 120;

// Determine required buffer size and allocate buffer
numBytes = avpicture_get_size(dst_fmt, dst_w, dst_h);
buffer = new uint8_t[numBytes + 64];

//put a PPM header on the buffer
int headerlen = sprintf((char *) buffer,
"P6\n%d %d\n255\n",
dst_w, dst_h);

_v->buf = (uchar*)buffer;
_v->len = avpicture_get_size(dst_fmt,dst_w,dst_h) +
headerlen;

// Assign appropriate parts of buffer to image planes
// in pFrameRGB...

// I use libswscale to scale the frames to the required
// size.
// Setup the scaling context:
SwsContext *img_convert_ctx;
img_convert_ctx = sws_getContext(
pCodecCtx->width, pCodecCtx->height,
pCodecCtx->pix_fmt,
dst_w, dst_h, dst_fmt,
SWS_BICUBIC, NULL, NULL, NULL);

// Read frames and notify
i=0;
while(av_read_frame(pFormatCtx, &packet)>=0)
{
// Is this a packet from the video stream?
if(packet.stream_index==videoStream)
{
// Decode video frame
avcodec_decode_video(pCodecCtx,
pFrame,
&frameFinished,
packet.data,
packet.size);

// Did we get a video frame?
if(frameFinished)
{
// Convert the image to RGB
sws_scale(img_convert_ctx,
pFrame->data,
pFrame->linesize,
0,
pCodecCtx->height,
pFrameRGB->data,
pFrameRGB->linesize);

emit frameReady();

//My video is 5FPS so sleep for 200ms.
this->msleep(200);
}
}

// Free the packet that was allocated by
// av_read_frame
av_free_packet(&packet);
}

// Free the RGB image
delete [] buffer;
av_free(pFrameRGB);

// Free the YUV frame
av_free(pFrame);

// Close the codec...

// Close the video file...
} //end VideoThread::run

Ok so I have a frame-grabber that emits a frameReady signal everytime the buffer is full and ready for painting.

A couple of things to notice:

• I convert the image format to PIX_FMT_RGB24 (avcodec.h), which is required by Qt's QImage::fromData() method.
• I scale the image using ffmpeg's libswscale. All conversion/scaling methods inside libavcodev are deprecated now.
  But it's fairly simple, here's a good example. Just remember you need a sws_getContext and then sws_scale.
• I totally disregard actual frame rate here, I just sleep for 200ms because i know my file is 5FPS. For a (far-) more sophisticated way to get the FPS, very important if this is not a constant frame-rate video, you can find here.
• I don't cover audio in this example, although the mechanism to extract it from the file exists... you just need to grabe the audio stream's frame. For playing audio you also need some Qt-external library. In a different project I used SDL very easily, here's an example online.

Now, for painting over the widget.
This is fairly easy:

void VideoWidget::paintEvent(QPaintEvent * e) {
QPainter painter(this);

if(buf) {
QImage i = QImage::fromData(buf,len,"PPM");
painter.drawImage(QPoint(0,0),i);
}
}

Two things to note:

• The widget needs to be given the pointer to the video frame buffer (buf).
• The frame buffer needs to be in a PPM format. That means it needs to get a PPM header, which looks something like this: "P6\n320 240\n255\n", and then all the pixels in 3-byte per-pixel format (RGB24). You can see that i take care of that in the previous code block.

Finally we need to orchestrate this whole mess.
So in my GUI-screen class I do:

....
vt = new VideoThread();
connect(vt,SIGNAL(frameReady()),this,SLOT(updateVideoWidget()));
vt->start();
....

And:

void playMessage::updateVideoWidget() {
videoWidget->repaint(); //or update().
}

This will make the widget repaint on each frame ready.

Note:

• In this example I don't take care of multi-threading issues. Since the GUI and the ffmpeg decoder threads share a memory buffer, I should probably have a mutex to protect it. It's a classic producer-consumer problem.
• Performance wise, Qt's paint mechanism is by far the worst way to go when displaying video... but it's great for a quick-and-dirty solution (I only needed 5fps). A more performance favorable solution will probably be using an overlay block and frame-serving with SDL.
  

Enjoy!
Roy.

Thinking of new services...

הלימודים עלינו

לא בטוח שלימודים בחינם זה הכוון הנכון...